Foreign Affairs and International Trade Canada

international.gc.ca

Common menu bar

• Français
• Home
• Contact Us
• Help
• Search
• canada.gc.ca

Breadcrumb

1. Home
2. >
3. About the Department
4. >
5. Accountability

Summative Evaluation of Young Professionals International (YPI)

(March 2006)

(PDF Version, 348 KB) ^*

Table of Contents

• Glossary of Acronyms and Terms
• Executive Summary
• 1. Introduction
  • 1.1 Background and Program Description
  • 1.2 Evaluation Issues and Objectives
  • 1.3 Methodology
  • 1.4 Methodological Limitations of the Evaluation
• 2. Evaluation Results
  • 2.1 Continued Relevance
  • 2.2 Success
  • 2.3 Alternatives/Leveraging
  • 2.4 Program Design and Delivery
• 3. Conclusions and Suggestions for Improvement
  • 3.1 Conclusions
  • 3.2 Suggestions for Improvement
• Appendix A - Management Response

Glossary of Acronyms and Terms

CIDA

- Canadian International Development Agency, whose mandate is to support sustainable development in developing countries in order to reduce poverty and contribute to a more secure, equitable, and prosperous world. The Agency's work is concentrated in the poorest countries in Africa, Asia, and Latin America.

DFAIT

- Department of Foreign Affairs and International Trade is the result of a recent (April 1, 2006) reintegration of the departments of Foreign Affairs and International Trade, which is intended to bring greater coherence in serving Canadian foreign policy and trade interests. As such, DFAIT will participate in the activities and address the Objectives of the two departments, as follows:

• Foreign Affairs Canada, representing Canada around the world through a network of embassies and trade and diplomatic offices, and by participating in multilateral institutions and international treaties and arrangements, in order to ensure Canadians' security within a global framework and promoting Canadian values and culture on the international stage.
• International Trade Canada, responsible for positioning Canada as a world business leader for the 21st Century, and working to promote prosperity by helping large and small Canadian companies expand and succeed internationally, by promoting Canada as a dynamic place to do business, and by negotiating and administering trade agreements.

GHY

- Young Professionals International, the unit of DFAIT that manages the Youth Professionals International Program (YIP).

HRSDC

- Human Resources and Social Development Canada

IYIP

- International Youth Internship Program, managed by Canadian International Development Agency (CIDA).

NetCorps

- The NetCorps Canada International program, delivered by Industry Canada and a coalition of nine non-governmental organizations, gives young people the opportunity to participate in innovative internships in the field of international development and information and communications technologies.

NGO(s)

- Non-governmental organization(s).

RMAF

- Results-based Management and Accountability Framework, which is a blueprint for results-based management of a program, policy, or initiative, which: provides information on what it is expected to achieve; describes its rationale, resources and performance logic; and presents the measurement, evaluation and reporting strategies that will be implemented to track progress

YES

- The Federal Government of Canada's Youth Employment Strategy, which helps young Canadians (aged 15 to 30) obtain career information, develop skills, find good jobs and stay employed, through a broad range of initiatives under three programs. Thirteen Government of Canada departments and agencies work with partners in other governments, businesses and communities to deliver the initiatives under these programs.

YPI

- The Young Professionals International Program.

Top of page

Executive Summary

Program Description

The major objective of Young Professionals International (YPI) is to facilitate the school-to-work transition of unemployed and underemployed Canadian post-secondary education graduates under 30 years of age into a career with an international focus, by assisting them to acquire advanced skills, knowledge and international work experience, and by preparing them to obtain and maintain employment.

Towards this end, YPI makes contributions to Canada-based Implementing Organizations that develop placements with overseas organizations/employers (Host Organizations) and that recruit eligible youth seeking first career-related international work experience to fill the placements. The Program is administered by the Young Professionals International (GHY) unit at the Department of Foreign Affairs Canada and International Trade (DFAIT), within the Human Security and Human Rights Bureau of the Department. YPI is a component of the Career Focus stream of the Government of Canada's Youth Employment Strategy (YES).

Annual YPI expenditures amount to $6.4M. On average, approximately 400 youth, and just over 40 Implementing Organizations and 300 Host Organizations participate in YPI per year. DFAIT contributes a maximum of $15,000 per placement to Implementing Organizations, with a minimum of $12,000 to cover participants' costs (language training, travel, health insurance, living expenses and accommodation), and a maximum $3,000 to cover administrative costs (the project coordinator's salary, recruitment and overhead).

Overall Conclusions

There is a need for a Program like YPI to provide the international work experience and employability skills that youth require to enter employment, which they would normally have difficulty obtaining. Partner organizations' and the participants' expectations about the Program have been met and YPI continues to address the priorities of the Department.

YPI has been successful in positively impacting on participants and DFAIT. After their placements, a majority of participants have the skills targeted by the Program and are employed up to two years following completion. However, the degree to which these outcomes have been sustained over the longer term could not be demonstrated. The Program is reaping benefits for the Department as well by helping the Department meet its priorities and by strengthening ties.

YPI is being delivered in effective fashion. Equity group representation among participants generally exceeds targets as well as proportions in the Canadian population. Safety and security measures of the Program are appropriate. To enable both skill measurement and acquisition, participants are provided with access to self-administered skills assessment and learning tools, the latter having been adopted by other similar federal programs. However, it is still too early to measure the effectiveness of these tools. No operational constraints were identified, other than partner organizations having difficulty dealing with the federal government's fiscal year and having to apply each year for funding.

No issues were raised regarding the amount provided by the Program to cover administrative costs, but some concern was expressed that the stipend to cover participants' costs is insufficient in some countries. There is a great deal of accountability and performance measurement taking place under the Program. Particularly laudatory is the YPI post placement survey, which is rarely employed in other programs. However, issues were raised around the ability to comprehensively measure skill attainment and longer-term employment outcomes using this survey.

A description of the evaluation Methodology and more detailed results follow.

Evaluation Issues, Objectives and Methodology

The evaluation addressed the Program's continued relevance, alternatives/leveraging, success in achieving intended results, and design and delivery, which are the evaluation issues raised in the YPI Results-based Management and Accountability Framework (RMAF) and refined in consultation with GHY and the evaluation unit of DFAIT. The evaluation took place between November 2005 and March 2006 and covered the period 2000 to 2005. To address the evaluation questions, the following methodologies were implemented:

• A review of YPI documentation, project files, and other Program documents, including results from YPI self-administered post-placement participant surveys;
• Interviews with 23 key informants, including representatives of GHY, Implementing Organizations, Host Organizations and overseas posts; and
• Case studies of five YPI participants comprising interviews with five youth.

Findings and Conclusions

Continued Relevance

Relevance to Needs: There continues to be a need for the Program. Respondents unanimously agreed, supported by the documentation and data reviewed for the evaluation, that there is a need for a program like YPI. Evidence of this includes the employer-desired employability skills that the Program is focused on; the limited opportunities for young people to obtain the kind of experience, knowledge and skills afforded by the Program; the fact that YPI targets, in addition to the unemployed, the underemployed, who represent relatively large proportions of students with arts or commerce degrees and about a half of the university graduates under 30 years of age; and the need for people with international skills owing to the outward orientation of Canada. Research from the U.S., a country similarly outward oriented, corroborates the importance of having employees with international skills. Further, many more organizations and participants apply to participate in the Program than funding will allow. Very few private sector companies apply for funding, and of these, the proposal success rate has been lower than for non-governmental organizations.

Met Expectations: Expectations about the Program have been met by all concerned. Reasons for participating in YPI identified by Implementing Organizations and Host Organizations include specific benefits to the organization (such as increasing staff capacity, or raising awareness of their cause) as well as a desire to help young people find work. Partner organizations expressed a great deal of satisfaction that their expectations were met. Specifically, these included new and strengthened existing partnerships, greater awareness of the organizations by the participants, and access to high-calibre participants. Participants profiled in case studies were pleased with their placement. Additionally, there is indication that views about Canadians are being positively altered by the YPI experience.

Alignment with DFAIT Priorities: YPI supports the priorities of DFAIT. According to YPI staff and management, the Program was explicitly designed to meet DFAIT priorities, and documentation and data reviewed demonstrate that the Program's activities are in accordance with those of DFAIT and YES.

Success

Benefits for Participants: The Program is having a positive impact on participants' skills and employment, indicating attainment of YPI's main objective. Up to two years following completion of their placements, large proportions of participating youth reported they had advanced and international skills and knowledge targeted by the Program, including inter-cultural, adaptability/flexibility, patience/tolerance, communication, research, language, organization, and networking skills. Also, a majority of participants reported either being employed or returning to school one to 24 months after their placements. Evidence of longer term impacts was not available.

Benefits for Department: YPI also benefits DFAIT. Former participants often work with the Department or in fields that benefit the Department. Other benefits to the Department were identified, such as "concretizing" its stated interest in benefiting youth, and allowing DFAIT to support non-governmental organizations (NGOs) and their work on issues (i.e., human rights) important to the Government. YPI participants were also said by YPI key informants to be very helpful in strengthening the linkages/networks between the embassy and companies/NGOs in the region especially for embassies that have small staffs; in effect, participants become "ambassadors" for Canada.

Alternatives/Leveraging

Alternatives: Few programs comparable to YPI were identified in key informant interviews as being offered to either Canadian or foreign youth. CIDA does offer a similar program, International Youth Internship Program (IYIP), but its focus is on development in developing countries. While YPI does place youth in some developing countries, it is primarily for purposes of strengthening human rights, democracy and international business in these countries, which are areas directly related to DFAIT's mandate.

Leveraging: The amount of financial and in-kind contributions that DFAIT is able to leverage from Implementing Organizations and Host Organizations varies widely depending on the resources of the organizations. The leveraging obtained by CIDA for IYIP was greater than that for YPI, but another somewhat similar program, Industry Canada's NetCorps Program, appeared not to have any leveraging.

Design and Delivery

Equity Group Representation: The representation of different equity groups among YPI participants closely matches or even exceeds targeted proportions, as well as population proportions. In every year, women exceed the 50 per cent target by a fair amount. Respondents identified some challenges with respect to YPI measures to ensure the appropriate representation of designated target groups, such as over-representation of women in the relevant educational disciplines (which may be why there is a larger proportion of women than men among YPI participants). Despite much encouragement by GHY for Implementing Organizations to place participants with disabilities, and despite partnering with the Canadian National Institute for the Blind for the period under review, youth with disabilities are somewhat underrepresented among YPI participants.

Safety and Security Measures: Safety and security measures of the Program are appropriate. All Implementing Organizations submitting proposals for Youth Employment Strategy funding, including that of YPI, must describe a safety and security strategy, including a contingency plan for participants that they will put into practice when they receive funding. Overall, impressions on the part of Implementing Organizations, Host Organizations and overseas posts of the safety and security measures implemented by YPI are positive. There was a sense among some Implementing Organizations that safety and security have improved over the years.

Skills Assessment and Learning Tools: Program participants are provided with access to skills assessment and learning tools, but it is still too early to measure their effectiveness. The self-administered on-line Skills Self-Assessment Tool permits participants to identify skills and learning gaps as well as progress, and Virtual Campus is a series of one-hour independent learning modules that participants may access via the Internet, CD-ROM or in print format. The skills assessment tool has been adopted by other YES programs. The fact that the assessment and learning tools have been up and running only recently (2004 and 2005, respectively) means that measurement of their impact to date cannot be made. While there is an on-line Virtual Campus evaluation form for participants to complete, use of it is voluntary and low.

Implementing Organization representatives have mixed opinions of the potential utility of the Skills Self-Assessment and Virtual Campus tools. Many feel that these tools are well-designed, but are not always practical, the two main issues being poor Internet access and the time needed to fill forms out.

Funding for Projects: No issues were raised regarding the amount provided to cover organizations' administrative costs (maximum of $3,000 per participant) but some concern was expressed about the stipend provided for participants' costs (minimum of $12,000). Implementing Organizations' concerns with the participants' stipend included the fact that the amount had not changed since the Program's inception, and that the cost of living varies greatly among countries.

Performance Measurement: There is a great deal of accountability and performance measurement taking place for the Program. These measures include administrative files and data maintained by GHY; regular performance reports submitted by GHY to YES; regular reports submitted by Implementing Organizations to GHY; and self-administered regular post-placement participant surveys conducted by GHY - all of which proved to be good sources of information for this evaluation. Other monitoring activities include site visits by GHY managers in the host country undertaken to ensure YPI's integrity, and audits of Implementing Organizations conducted by the Department.

However, some gaps were identified in performance measurement. Shortcomings of the post-placement surveys include potential response bias (e.g., mainly those with positive or negative experiences participating in the survey), and a lack of long-term measure of participant impacts owing to poor response to the 24-month survey and there being no surveys past that point. As well, under the terms of the agreement with the YES, GHY is forbidden from maintaining participants' Social Insurance Number, which would have facilitated tracking participants following their placement. There is also a lack of quantitative data on impacts on partner organizations (Implementing and Host Organizations) which could be a good additional source of information on participant outcomes. Although most Implementing Organization interviewees indicated that they are satisfied with YPI monitoring and reporting activities, the quality and timeliness of the information provided in their reports vary greatly from one organization to another.

Operational Efficiency/Constraints: YPI staff and managers feel that there are no operational constraints faced by the Program, pointing instead to a number of factors contributing to Program efficiency. These positive elements include a dedicated unit within the Department staffed with people who are familiar with the Program, the ability to call upon other elements of the Department including the services of overseas posts, and processes that are spread evenly throughout the year. Implementing Organizations and overseas posts did not have any major suggestions for alternate delivery models.

On the other hand, most Implementing Organization representatives identified one particular issue: the adherence to single federal government fiscal year funding. This was problematic because, for example, it did not correspond to the fiscal year of either universities or Host Organizations. Organizations that were regular Program partners found it burdensome to apply for funding year after year. A variety of other constraints were identified, but by only one or two individuals each.

Recommendations for Improvement

On the whole, there is evidence of a continued need for YPI, the Program is operating efficiently, and there is evidence of positive impacts for the Department and for participants, at least in the short term. However, the Program could benefit from making some minor adjustments. On the basis of the YPI evaluation findings, the following key recommendations are made to DFAIT:

1. Reconsider the current one-year funding delivery arrangement. The YPI fiscal year does not always correspond to the fiscal year of participating organizations. For example, Host Organizations frequently do not operate on the federal government's fiscal year basis (April 1 to March 31) and universities (as Implementing Organizations) have difficulty recruiting participants in time to allow for a six-month placement before the end of the year. "Proven partners" find it difficult and costly to write a proposal every year. GHY should examine whether funding can be delivered in a manner that is more flexible to Implementing and Host Organizations' needs, and consider the possibility of multi-year funding agreements.
2. Assess the adequacy of participant stipends in various locales. A variety of issues were identified with regard to how much money is given to participants during their placements, regardless of country, ranging from the cost of living in Western European countries to the cost of airline tickets and medical necessities in more "exotic" locales. Therefore, GHY may want to consider investigating what the relative cost of living is in each placement and whether the $12,000 participant stipend is a realistic amount for their subsistence in each location.
3. Strengthen efforts to collect data on the use/effectiveness of the Skills Self-Assessment and Virtual Campus Tools. Many Implementing Organization representatives interviewed said that they have not kept track of how many participants use these tools. Indeed, during the course of this evaluation there was some difficulty getting useful evaluative information on participants' use and views of these tools. A future evaluation of the Program would benefit from having more data on the utility and popularity of these tools, possibly by making stronger efforts to encourage participants to make greater use of the skills assessment tool and the Virtual Campus on-line evaluation form.
4. Ensure collection and synthesis of performance information from partner organizations. While Implementing Organizations do regularly submit performance reports, the information is not synthesized and analyzed by GHY. Furthermore, Implementation Organizations are required to obtain performance information from their Host Organizations via a questionnaire addressing such issues as the level of compatibility between participants' skills and Host Organizations' needs, and their satisfaction with the process of arranging placements with the Implementing Organization. There is, however, limited evidence of Host Organizations completing these questionnaires. If such information was collected on a regular basis and stored in a database, it would be a good source of data on participant and Host Organization performance. GHY should consider the on-line Host Organization survey model used for CIDA's International Youth Internship Program.
5. Remedy shortcomings of post-placement data on participants. There are a number of issues with the post-placement surveys that have inhibited performance measurement, including the potential for response bias; an inability to measure skill gains (because a comprehensive pre-placement measure of skill levels is not available); imprecise questions on entry into the international employment; low response rates particularly in the 24-month survey (despite participants' signing letters of agreement with the Implementing Organizations to participate in the surveys); and a lack of longer-term measure of success. To increase response rates and the ability to measure long-term outcomes, remedies include: having GHY taking measures to maintain up-to-date contact information on participants by sending post cards annually asking participants to update their coordinates and participate in the post-placement surveys; ideally adding surveys beyond the 24-month mark; and strongly urging HRSDC (as administrator of the YES) to permit the collection of participants' Social Insurance Number to facilitate tracking them down for purposes of measuring post-placement outcomes. To increase precision, consider asking questions on skill levels at the start of their placement in the six-month post-placement survey and on the "international-ness" of the jobs they occupy in all post-placement surveys.

Top of page

1.0 Introduction

• 1.1 Background and Program Description
• 1.2 Evaluation Issues and Objectives
• 1.3 Methodology
• 1.4 Methodological Limitations of the Evaluation

1.1 Background and Program Description

The major objective of the Young Professionals International (YPI) Program is to facilitate the school-to-work transition of Canadian post-secondary education graduates to a career with an international focus, by assisting graduates in obtaining advanced skills, knowledge and international work experience, and by preparing them to obtain and maintain employment.

Towards this end, the YPI makes contributions to Canada-based Implementing Organizations that develop placements with overseas organizations/employers (Host Organizations) and recruit unemployed and underemployed eligible youth seeking a first career-related international work experience to fill the placements. It is administered by the Young Professionals International (GHY) unit at the Department of Foreign Affairs Canada and International Trade⁽¹⁾ (DFAIT), within the Human Security and Human Rights Bureau of the Department. YPI was initially known as the Youth International Internship Program.

YPI is a component of the Career Focus stream of the Youth Employment Strategy (YES). YES was launched in 1996 to address challenges faced by young people making the transition from school to the workplace. YPI is a horizontal initiative led by the Human Resources and Social Development Canada (HRSDC) with participation from other federal departments and agencies including the Canadian International Development Agency, Industry Canada, Environment Canada and Canadian Heritage.

DFAIT contributes a maximum of $15,000 per placement to the Implementing Organization. A minimum of 80 per cent (or $12,000 per placement) must be used for participant costs such as language training, participant travel and health insurance, visas, immunizations, living expenses, accommodations, briefings and debriefings. A maximum of 20 per cent (or $3,000 per placement) may be used by the Implementing Organization to cover some or all of the actual and reasonable administrative costs of the project, such as the project coordinator's salary and mandatory employment-related costs, office telecommunications, recruitment and overhead.

Annual Program expenditures amount to $6.4M (Table 1.1). At $510,000, salaries and operating and management represent about eight per cent of total Program expenditures. Note that in 2006-07 and 2007-08, it is expected that $1,125M will be taken back by the Department under the Expenditure Review initiative of the federal government. Delivery of the Program involves five full-time equivalent employees (FTEs).

Source: YPI Results-based Management and Accountability Framework and additional information provide by GHY.

Program data indicate that participation in YPI has remained fairly steady since 2000-01 (Table 1.2). Over the period in question, on average, just over 400 youth, 40 Implementing Organizations and 300 Host Organizations participated in YPI per year. Note that YPI is typically the beneficiary of funds re-allocated from other YES programs not expecting to spend all their funds for the fiscal year, which explains any excess over 385 YPI participants in a year. However, in 2005-06, this did not take place. which is why the number of YPI participants was lower than in previous years. On average, GHY provides funding to 9-10 participants per Implementing Organization per year, each of which places participants in 6-7 Host Organizations. An average of 1.3 participants were placed in each Host Organization.

Source: Program data - reports submitted by GHY to YES and data provided by GHY.

During the Program's first seven years (1997 to 2004), a total of 3,215 participated in YPI projects⁽²⁾. Over that period, about 100 Implementing Organizations arranged placements with about 1,580 overseas Host Organizations, in over 140 countries.

Program data further indicate that Implementing Organizations are predominantly academic or non-governmental organizations. Over the period 2000-01 to 2005-06, almost seven in ten projects (69 per cent, or 181 projects) were sponsored by non-governmental organizations (NGOs), 29 per cent (76 projects) by universities or colleges, one per cent (or three projects) by private sector companies, and one project by a band council.

Table 1.3 presents a profile of participants in the Program. The results indicate participants are on average 25.5 years of age, with about three-quarters between the ages of 23 and 29 years (24.9 + 27.3 + 24.1 = 76.3 per cent). Almost two in three participants (63.8 per cent) have a Bachelor's degree, with one in four having completed a Master's or PhD (24.9 per cent). A placement lasts on average 6.5 months, with about four in five placements (81 per cent) lasting six months exactly. About a third of participants are placed in Europe (35 per cent), with similar proportions placed in each of Africa, North America and Asia (19-15 per cent).

Source: Program data provided by GHY.

The provincial/territorial composition of YPI participants reflects fairly well that of the Canadian population (Table 1.4), based on Census data from Statistics Canada.⁽³⁾

* Excludes Census data for one or more incompletely enumerated Indian reservers or Indian settlements.

Source: YPI program data provided by GHY and Census data from Statistics Canada

1.2 Evaluation Issues and Objectives

This summative evaluation of YPI was carried out in response to a Treasury Board condition of renewal of YES funding to the Program. The evaluation addressed the issues of continued relevance, cost effectiveness, success in achieving intended results, and design/delivery issues. These evaluation issues and questions were developed using questions identified in the Terms of Reference (ToR) for this evaluation project. The questions were refined and indicators and data sources developed in collaboration with a representative of the Evaluation Division of the Inspector General's Office of DFAIT (Project Authority) and with YPI Program staff and management. This evaluation took place from November 2005 to March 2006, covering the period 2001-02 to 2005-06.

1.3 Methodology

The Methodology for the summative evaluation of the YPI Program was composed of multiple lines of evidence, as follows:

• A review of YPI documentation, project files, and other Program documents, including results from YPI self-administered post-placement surveys;
• Interviews with key informants; and
• Case studies of YPI projects.

a) Review of Program Documents, Project Files and Data

A review of available YPI Program documentation and files was conducted to address all evaluation issues. Program documentation and data reviewed included:

• basic Program documents such as: Treasury Board submission, Terms and Conditions, proposal funding criteria, application forms; contribution agreements, the 2003 YPI Results-based Management and Accountability Framework (RMAF), and budget and expenditure information;
• 20 project files from 2003-04 and 2004-05, which include proposals and reports from implementing organizations;
• onsite visit reports (internal);
• results from post-placement surveys of participants conducted by GHY, 2000 to 2004;
• results of post-placement studies conducted by individual Implementing Organizations (n=2);
• Connect and Contribute - A Seven-Year Report of YPI (2005), which provides, among other things, profile information on Implementing Organizations;
• YPI Reunion Interviews (anecdotal evidence on participant experiences/outcomes);
• YPI internal audit report (2005);
• YES evaluation report and literature review, 2005;
• reports of evaluations of Industry Canada's and Canadian International Development Agency's (CIDA) youth internship programs (Netcorps and International Youth Internship Program);
• other Program documents and data obtained from GHY, such as reports to the YES, applicant counts as well as participant and Implementing Organization profile data;
• Statistics Canada data, mainly from the Census, to compare the participant profile to; and
• Results from Canadian and US research on skills and employment needs of youth (see next chapter on evaluation findings for details on sources).

b) Key Informant Interviews

In general, key informant interviews help in gaining a better understanding of the perceptions and opinions of individuals who have had a significant role or experience in the design and/or delivery of YPI, or who have a key stake in the Program.

We conducted interviews with 23 key informants. Interview questions were open-ended in order to allow the interviewees to explain their responses in depth and detail. Interviews were between 20 and 60 minutes in duration, and were conducted in English or French, depending on the preference of the respondent. Most were conducted by telephone; one interview was conducted face-to-face.

Interview respondents were identified in cooperation with the Project Authority and with YPI Program staff and management. These individuals generally had a thorough understanding of various issues around the relevance, effectiveness, and delivery of the Program. The key informants comprised the following:

• Assistant Deputy Ministers (n=2);
• YPI Program Managers and staff (n=2);
• Representatives of overseas posts (n=4);
• Representatives of Implementing Organizations (n=7); and
• Representatives of Host Organizations (n=7).

c) Case Studies

We conducted five case studies of YPI projects dating from 1999 until the present. Data collection for the case studies consisted of an interview with YPI participants and a brief documentation review. The purpose of this methodological component was to obtain opinions, perceptions and observations on the experience of participants, with an emphasis on outcomes and experiences since their placements. This qualitative evidence was gathered to add flavour to, supplement and assist with the interpretation of other lines of evidence of success, but should not be considered representative of all placements.

Potential cases were identified from suggestions provided by the Implementing Organizations and GHY. Selection criteria utilized were such that the case-study projects include at least two men and at least one francophone; a former participant currently working for DFAIT; and an Aboriginal participant. Regional representation within Canada is also a consideration, and only one participant per Implementing Organization was considered. This selection was conducted under the guidance of the Project Authority, as well as Program staff and management.

1.4 Methodological Limitations of the Evaluation

Two main limitations to this evaluation are noted. First, as prescribed in the original Request for Proposals for this evaluation, qualitative information was collected from a small number of Implementing Organizations and Host Organizations, which play an important role in this Program. While the qualitative evidence, supported by a review of the project files of the Implementing Organizations, does provide some illustrative detail, a more comprehensive survey-based data collection approach would have provided more definitive information on delivery and success issues for partner organizations, and potentially for participants.

Second, the YPI post-placement participant surveys, which served as the basis for measures of impact on participants in this evaluation, had two main shortcomings. First, the measures of skill attainment are self-reported by participants (which may have introduced certain biases) and there was no measure of skill levels at the start of placements, thus precluding measurement of skill gains.⁽⁴⁾ Second, measuring long-term employment outcomes using the survey was not possible, given the fact that there were survey data only up to two years following a placement, and, even here, sample attrition was a problem, as a very small proportion of the original participants responded to the 24-month surveys.

Top of page

2.0 Evaluation Results

• 2.1 Continued Relevance
• 2.2 Success
• 2.3 Alternatives/Leveraging
• 2.4 Program Design and Delivery

The results for each of the evaluation issues - continued relevance, success, alternatives/leveraging, and design and delivery - are presented in this chapter.

2.1 Continued Relevance

a) Need for Program

The reasons Implementing Organizations and Host Organizations identified for participating in the YPI Program were varied. A slight majority mentioned specific benefits to the Host Organization (e.g., having individuals with English language skills, increasing staff capacity or needing flexible staffing, filling recruitment needs) or the Implementing Organization (e.g., strengthening existing partnerships with Host Organizations, and promoting their own Objectives by raising awareness of their cause). Other reasons cited include a desire to help young people find work (mentioned by an Implementing Organization) and positive impressions of the Program (mentioned by a Host Organization).

Most key informants believe there is a need for the Program, specifically to address access issues. When asked whether there was a need for a program like YPI, representatives of overseas posts and Implementing Organizations unanimously responded in the affirmative. Most cited limited opportunities for young people to get the kind of practical work experience and knowledge afforded by the Program, which a post-secondary education typically does not provide. For example, one respondent stated that "You have to have work experience on the ground to have credibility." Indeed, it is often heard that "one cannot get a job without work experience or get experience without a job." YPI, and programs like it, address this gap by providing the needed work experience that post-secondary education graduates often cannot obtain. A glance through employers' job advertisements would confirm work experience as a requirement for jobs. According to some people, this is particularly the case for certain geographic or cultural groups of Canadians (i.e., Atlantic Canadians, Aboriginal persons). Research confirms that those interested in international careers face particular difficulties, such as substantial costs of in-the-field training⁽⁵⁾. In addition, international organizations typically maintain quotas for the maximum number of people they will hire from any member country.

Furthermore, research confirms the need to address, as the Program does, the issue of underemployment, considered to be an under-utilization of human capital for the nation.⁽⁶⁾ A recent Statistics Canada study found that three in ten university graduates (30 per cent) with a strong attachment to the workforce⁽⁷⁾ occupied a job for which they were overqualified (requiring only a high school certificate) for at least a month during a six-year follow-up period. Overqualification incidence was particularly high for those with a degree in commerce (37 per cent) and in arts, humanities and social sciences (32 per cent), which are the fields that the majority of YPI participants complete studies in. Accordingly, the Program gives such graduates a significant advantage over others by providing them with international "professional" work experience. Moreover, the incidence is particularly high (48 per cent) for those in the age bracket targeted by YPI (under 30 years of age).

Furthermore, documents reviewed for this evaluation support the view that YPI is focused on the skills youth need. It was stated that three essential competencies are targeted in preparing globally-minded youth for employability: intercultural, technical/professional specialization, and personal.⁽⁸⁾ The Results-based Management and Accountability Framework (RMAF) for YPI states that international work experience opportunities, particularly those with mentoring components, will provide skills development related to the first cross-cultural competency (i.e., cross-cultural communications, heightened ability to adapt quickly to changing environments) in a low-risk environment. Furthermore, skills developed through YPI participation are consistent with many of the 11 main skills employers are indicated as needing: communication, managing information, problem solving, demonstrating positive attitude, being responsible, adaptability, working in teams, participating in projects.⁽⁹⁾

U.S. research corroborates the importance of having employees with international skills.⁽¹⁰⁾ Although this is U.S.-based evidence, the same ideas would also apply to Canada, as the two countries' economies are very similar and there is considerable trade taking place between them. According to a survey of 111 U.S. companies, the following was found:

• Almost 80 per cent of the companies would expect their overall business to increase if more international expertise were available within their staff;
• Appreciation for cross-cultural differences is the most important international skill for both professional staff and line management; and
• Almost 80 per cent of the firms indicated they will place a greater emphasis on international competence among management and employees over the next ten years.

The need for people with international skills is further demonstrated through economic and diplomatic evidence. The RMAF states that for Canada to succeed in the global, knowledge-based economy, it must be capable of producing a critical mass of well-educated and highly skilled people. It was also said to be critical to provide opportunities for young Canadians to develop, ideally through work experiences, the skills specific to functioning in an increasingly globalized economy. The most recent National Accounts data (fourth quarter of 2005) demonstrate the importance of exports to the Canadian economy, as exports represent 39 per cent of Gross Domestic Product of Canada with over $450 billion of goods exported in 2005.⁽¹¹⁾ This suggests a high degree of dependence on foreign trade and thus a need for people with international skills. Also, Canada is a member of numerous international organizations⁽¹²⁾ and thus is in need of preparing young people with international skills to work in these organizations and contribute to strengthening Canada's place on the international stage. Finally, interview respondents from some Host and Implementing Organizations stated that Canada, particularly recently, has suffered from a diminishing profile in the international sphere and therefore qualified youth, such as YPI participants, are needed to help improve its image in the world.

The volume of applications from both potential and accepted participants and Implementing Organizations demonstrates the level of interest in the Program. YPI data received from GHY indicate that, of the 5,219 youths for whom organizations applied to seek placements under the Program from 2000-01 to 2005-06, only about a half (47 per cent or 2,444) were actually placed because funding was obtained.⁽¹³⁾ Recall, as well, that YPI typically receives funds re-allocated from other YES programs to enable placement of participants over the 385 count. Similarly, of a total of 445 proposals submitted by Implementing Organizations to the Program, 262 were accepted, amounting to a 59 per cent success rate. This means that 41 per cent of organizations that sought funding from the Program over that period were turned away, suggestive of excess demand and interest in the Program on the part of organizations.⁽¹⁴⁾

On this point, it should be noted that, because of limited funding and the need to manage expectations, GHY does not aggressively promote the Program to potential implementing organizations or to youth participants. For example, from 2000 to 2005, GHY representatives very rarely attended job fairs, except for one annually in Quebec.

Analysis of data on Implementing Organization by organization type reveals low uptake by and acceptance of applications from private sector organizations. Between 2001-02 and 2005-06 period, about three per cent (n=11) of proposals were submitted by private sector organizations, and of these proposals, 27 per cent were successful in obtaining funding. This compares to a "success rate" of 67 per cent for NGOs and academic organizations submitting proposal to YPI. The reason for the lower success rate among private companies is not known.

It is also important to note that many of the organizations obtaining YPI funding have done so on more than one occasion. The data indicate that, of the 445 proposals submitted to the Program from 2000-01 to 2005-06, 41 per cent were from different organizations (n=186); about a third (32.1 per cent) of the 262 successful proposals were from different organizations (n=84). In other words, on average, an organization received funding 3.1 times (262 / 84) over that period. Of the 84 organizations accepted for funding over the six-year period, 24 (29 per cent) received funding once, and a similar number (25) received funding five or more times.

b) Extent to which Expectations are Being Met

Implementing Organizations and Host Organizations interviewees generally are happy with the degree to which their expectations of the YPI Program have been met. (As discussed previously, there was a fair amount of variation in what their expectations actually were.) Evidence of this included: new and strengthened existing partnerships, greater awareness of organizations among participants, and high levels of satisfaction with the calibre of participants. In a few cases, Host Organizations hired their YPI participants after their placements were completed. The only major complaint from one organization was that the placements were too short, and that there should be more "continuity."

Views on Canada are being positively altered by the YPI experience. Host Organization representatives were asked whether or not participating in YPI had changed their perceptions of Canadians and Canadian values.⁽¹⁵⁾ While some said that their perceptions were not changed simply because they had worked with Canadians before and/or they had good impressions to start with, many described positive impressions they or their foreign colleagues had of Canadians as a result of YPI participants. For example, one Host Organization representative had previously thought Canada was weak in some areas of education such as marketing, but her/his perception changed as a result of the YPI experience. Another commented that YPI participants have demonstrated Canadian commitment to humanitarian issues as well as capacity to provide appropriate response and support mechanisms to international organizations.

Case study subjects generally feel that their expectations about the YPI experience were met. They all feel that, for the most part, they got the experience they desired. A reservation was raised by one subject who had hoped to be more immersed in the local culture, rather than being housed in a compound with other foreigners to the host country.

c) Alignment with Departmental/ YES Priorities

There was unanimous agreement among all those consulted on this issue that YPI supports DFAIT priorities. According to YPI staff and management, the Program was explicitly designed to meet DFAIT priorities. Implementing Organizations' proposals are required to state why DFAIT should support the proposed project and how DFAIT priorities would be reflected in the projects if funded (statements from one of the Implementing Organizations corroborate this assertion). The YPI selection committee reviews all potential applications and considers to what extent Departmental priorities are supported, thereby ensuring that issues such as human rights, peace building, prosperity, and promoting relations with the U.S. are addressed. Points are awarded to proposals proportionately according to the degree to which they meet Departmental priorities. Also, the experience, international perspective, and outward orientation gained by participants is expected to help Canadian companies that hire these participants become more export-ready, which is a goal of the Department with respect to international trade.

Documentation reviewed demonstrates that the Program's activities are in accordance with DFAIT's priorities. The fact that YPI addresses the Department's three foreign policy priorities can be demonstrated by the alignment of funded projects with these priorities, as the following indicates (1997 to 2004⁽¹⁶⁾):

• Security: ensuring Canada's security remains a fundamental responsibility of government and a key foreign policy priority. Over the first seven years, 1,322 YPI participants (41 per cent of the 3,215 participants) worked in areas related to security, such as conflict prevention and resolution, children's rights, de-mining, democratic development, elections, human rights, health, international law, migration and population, small arms proliferation, weapons of mass destruction, terrorism and women's rights.
• Identity: Over the first seven years of the Program the department has implemented 783 placements (24 per cent) related to cultural industries and Canadian values, cultural media, distance and international education, indigenous issues, journalism, francophone language and culture, multicultural/intercultural relations, museums, performing arts, film and television.
• Prosperity: Canada's prosperity depends on the ability of Canadians to succeed in an increasingly globalized business world. In the first seven years of operations, YPI dedicated 1,110 placements (34 per cent) to ensuring that participants were given opportunities to develop critical international trade skills. Placements have also covered a wide range of trade issues, including business, economic and export investment development, environmental industries, information technology, market research, science and technology, sustainable development, technical assistance, tourism, trade mission logistics and trade policy.

As well, most project proposals refer to meeting the Department's priorities. In a review of the project files of 20 funded Implementing Organizations (2003-04 and 2004-05), it was observed that 17 Implementing Organizations included specific reference in their proposals about how their projects would contribute to DFAIT's mandate or Objectives.

More generally, according to YPI key informants, placements help strengthen the Canada "brand" and provide cultural and/or with trade "bridges to Canada". Certain other key informants (Implementing Organization and Overseas Post representatives) agreed with this, making such comments as "it's a means through which we're demonstrating our commitment to expanding linkages with institutes in many countries" and "it's really helped encourage Canadian participation in the international arena and to encourage and disseminate Canadian values abroad."

The Program's Objectives also align with the Objectives of Canada's Youth Employment Strategy (YES). The Government of Canada created the Youth Employment Strategy to help young Canadians (aged 15 to 30) obtain career information, develop skills, find good jobs and stay employed. According to websites for the YES and YPI, the Program is designed specifically to link with YES Objectives.

2.2 Success

a) Skills Impacts for Participants

Before presenting results, it is useful to review what information on skills is collected by the Program. There are three sources of such information.

• Implementing Organizations' project files: Implementing Organizations are required to identify in their proposals the skills which YPI participants are expected to develop in the proposed placement. The skills actually developed by participants are often described in an explicit section of the final reports submitted by Implementing Organizations, but sometimes a description of skills is integrated into the text of success stories where skills growth was implied (or not stated at all). The specific skills targeted are those that are likely to increase the employability of participants in an international context.
• Skills Self-Assessment Tool: Participants are asked to complete skills assessments prior to, during and after their placements. The primary purpose is to enable participants to develop and implement their own skills development plan and to measure progress in skills attainment. In situations where a sufficient number of participants use this tool at specific Implementing Organizations and Host Organizations, organizational trends may be examined with respect to skill development. Note that this tool has not been in use long enough yet to enable performance measurement (this issue is addressed in the Design and Delivery section, below).
• YPI post-placement surveys: Participants are asked to complete post-placement surveys on-line (six, 12 and 24 months following completion of their placement). These surveys include questions related to skill levels. (Other questions in the survey are asked on participants' current employment status, and the degree to which the YPI internship helped with post-placement job search and with acquisition of knowledge about Canada.) The results from these self-administered surveys are the basis for the findings below.

It should be noted that Implementing Organizations administer the post-placement surveys to participants. According to the YPI Passport document, it is the responsibility of these organizations to inform participants of the importance of completing the surveys in order to measure effectiveness of the YPI experience, as well as to make completion of the surveys a part of their agreement/contract with participants. Then, at the appropriate intervals (6, 12 and 24 months following placement completion), GHY sends an e-mail reminder to each of the Implementing Organizations, which in turn forward it to their former participants, who then visit the website to complete the survey. Thus, while participants do sign agreements with Implementing Organizations to participate in the post-placement surveys and the organizations are obliged to encourage the participants to do so, they are not strictly legally obligated to respond, and, in fact, as indicated below, a number do not. However, it would be difficult to enforce this. (The utility of these surveys for performance measurement is addressed further in the appropriate design and delivery section (Section 2.4e) below.)

Based on results from the six-month post-placement surveys from 2000 to 2004, large proportions of participating youth said they possess high levels of targeted skills following completion of their placements. The skills most frequently enhanced, as reported by participants, are inter-cultural effectiveness, adaptability/flexibility and patience/tolerance. These skills were consistently rated as "greatly enhanced" by approximately 50 per cent of the respondents in four out of the five years. This result was confirmed by an examination of the Implementing Organizations' reports to YPI. The review indicated that several skills were consistently developed over diverse Implementing Organizations and Host Organizations, but particularly those related to cross-cultural communication, research, language, organization, networking, and flexibility/adaptability skills.

To illustrate participants' level of attainment, results from the 2002 cohort's six-month post-placement survey (to which response has been the greatest) are presented in the exhibit on the next page. The exhibit indicates not only that almost half of participants reported that inter-cultural effectiveness, adaptability/flexibility and patience/tolerance had been greatly enhanced (indicating five on the five-point scale), but also that, on all skills, about half or more of the participants reported four or five on the scale. This suggests that participants believe their YPI placements played a significant role in the improvement of all skills examined.

Finally, two things should be noted about these surveys. First, the 2002 results are fairly representative of the results achieved over the five years during which post-placement survey responses were examined. Second, these results were confirmed in the review of the Implementing Organization files. The review indicated that several skills were consistently developed over diverse Implementing Organizations and Host Organizations, including particularly cross-cultural communication, research, language, organization, networking, flexibility/adaptability skills.

b) Employment, Education and Unemployment Impacts for Participants

A majority of participants report being either employed or returning to school at the conclusion of their placements. This finding is based on information collected from a sample of 20 year-end reports that Implementing Organizations submitted to GHY, 10 each in 2003-04 and 2004-05. Of the 17 project files with complete post-placement information on employment outcomes for each participant, which involved a total of 182 participants, 94 (52 per cent) were employed, 18 (10 per cent) returned to school, and 70 (38 per cent) were seeking employment. From 10 to 25 per cent of participants returned to school. A few Implementing Organization interviewees identified two potential reasons why participants returned to school:

1. the YPI experience had changed participants' priorities and outlook on life and they subsequently decided to change career direction, or
2. participants had planned to return to school all along and that the YPI experience was just the experience they needed or wanted before pursuing graduate studies.

The incidence of employment among former participants increases with time since completion of their placement. This conclusion is based on results from post-placement surveys that participants filled out on-line from 2000 to 2004 (results for the 2005 cohort were not available for this evaluation as the due date for submitted surveys had not been reached). The incidence of full-time/long-term (FT/LT) employment six months after the placement was in the 36-53 per cent range, compared to about 20 per cent at the start of the placement (in the six-month post-placement survey, participants are asked about their employment situation going into the placement). By 12 months, the proportions rose to 49-66 per cent.⁽¹⁷⁾ Note that this is likely true of youth generally as begin their careers and make their way in the work world. Over the period under consideration, the proportion of YPI alumni in school and in part-time/short-term jobs remained relatively steady, accounting for approximately 10-24 per cent of participants each year.

The exhibit on the next page illustrates, for the 2002 YPI cohort⁽¹⁸⁾, the increase in the incidence of employment, specifically FT/LT employment, over the post-placement period, and the decline in unemployment. The results show that the FT/LT employment rate rose from 14 per cent going into the placement to 36 per cent six months post-placement, 49 per cent 12 months post-placement, and 55 per cent 24 months post-placement (noting that the last results are based on a small number of observations).

There is some information that YPI participants are entering the international labour market following completion of their placements. From 2001 to 2004 between 50 and 65 per cent of YPI alumni indicated in the six-month post-placement survey that they attempted to find employment in the international labour market (data from 2000 is not included due to a low response total of 14 participants). When asked whether participants believe their YPI experience facilitated entry into the international labour market, between 33 and 38 per cent of total respondents indicated that their experience contributed positively.⁽¹⁹⁾

In the case study interviews with five YPI alumni, all indicated that their YPI experiences had been a factor to their current employment. Expressions illustrating the positive role played by YPI include: "pretty major", "huge factor", "allowed me to obtain position", "helped clarify my direction", and "very important, it really influenced my choices".

A few Individual Implementing Organizations have conducted participant surveys providing evidence of longer-term outcomes of the Program. However, because these results are for participants of just two organizations, and for other reasons indicated below, this evidence is illustrative and not necessarily representative of the outcomes for all participants. The results from the two surveys are as follows:

• The International Institute for Child Rights and Development (IICRD), based on qualitative evidence gathered in interviews with 15 of its past YPI participants (some of whom participated up to seven years previously), found that most YPI alumni (13 or 14) said they are working in areas related to their placement, most said the placement helped them to get where they are now, several said they had a positive impact on the community where they were placed, and ICRD feels that participants help in terms of strengthening relationships with both Host Organizations and subsequent YPI placements.⁽²⁰⁾ A number of the seven YPI participants from the 1990s are currently engaged in employment or educational activities with an international "flavour."
• The United Nations Association in Canada (UNA-Canada) surveyed 109 past participants (some of whom participated up to eight years previously) in YPI and CIDA's IYIP.⁽²¹⁾ The results indicate that large majorities (80-90 per cent) of YPI and IYIP alumni reported that their placement contributed to the development of their professional skills and their awareness and understanding of world affairs. Most felt they made a contribution to their workplace and helped promote Canadian values of tolerance of ethnic and religious differences.

Finally, YPI participants occupy a range of jobs following their placements.⁽²²⁾ These jobs include: marketing officer, constituency assistant, program manager, coordinator, communication and research assistant, senior marketing manager, communications coordinator, coordinator of volunteer development, web designer, marketing manager, researcher, and media auditor.

c) Benefits for DFAIT

YPI benefits DFAIT. There was the general consensus among key informants interviewed for this evaluation. Departmental Assistant Deputy Ministers and YPI managers/staff indicated that a large number of former YPI participants subsequently joined the Department (including Canada's Foreign Service), with an estimated 50 to 70 working within DFAIT at any given time. In a few instances, when returning to Canada as a result of their YPI experience, participants have even started NGOs that are helping to further causes for developing countries related to the priorities of DFAIT such as the promotion of human rights and international business. (See the section on Relevance which showed how YPI projects aligned with Departmental priorities.) It was further pointed out that YPI alumni who work in foreign business effectively become salespeople for Canada -- which benefits foreign trade Objectives of the Department. YPI participants who end up working in foreign organizations promote Canadian interests abroad by promoting Canada's place in the world as well as enhancing trade between the respective countries. This view is supported by the representatives of some Host Organizations, who pointed out that working with YPI has reinforced or enhanced their positive view of Canadians and Canadian values. Even participants who work in domestic organizations will have outward orientation/experience, which benefits the organization internationally, and makes it potentially more attractive to investment partnerships with foreign companies.

Other benefits to the Department were identified in the key informant interviews. It was said that having a program like YPI "concretizes" the Department's stated interest in benefiting youth. It was also said that YPI allows DFAIT to support NGOs and their work on issues (e.g., human rights) important to the Government that it cannot address directly. A majority of Implementing Organization interviewees indicated that YPI has strengthened connections between the Department and themselves. YPI participants were also said by YPI key informants to be very helpful in strengthening the linkages/networks between the embassy and companies/NGOs in the region, especially for embassies that have small staffs; in effect, participants become "ambassadors" for Canada.

One key informant indicated that more could be done to ensure that participating Host Organizations are in countries that reflect the Canadian government's interests such as the promotion of human rights and international business.

2.3 Alternatives/Leveraging

a) Existence of Other Programs

In general terms, no programs comparable to YPI were identified in key informant interviews as being offered to either Canadian or foreign youth. One Implementing Organization interviewee stated "everywhere I go countries ask me how they can do something similar; they think it's (YPI) fabulous. A lot of countries just don't have the government funds."

Still, many international programs exist for youth in Canada, though none serves precisely the same purpose and have the same structure as YPI. As noted, CIDA does offer a similar program, International Youth Internship Program (IYIP). The main difference between YPI's placements and CIDA's placements is the subject matter. CIDA's placements are "development" placements, i.e., in development work. Conversely, whereas YPI does have placement in developing countries, they would be in support of human rights, good governance, legal affairs, and international business, which are areas directly related to DFAIT's mandate.. Some of the YPI-funded Implementing Organizations are involved in development work and therefore have placements through both YPI and IYIP.

Other potentially comparable international internship programs identified in the key informant interviews with representatives of Implementing Organizations and Host Organizations include those provided by the following types of organizations:

• universities and colleges such as the University of British Columbia, Capilano College, Ryerson University;
• government: programs mentioned include CIDA's IYIP, Canada Corps, NetCorps, the UN Junior Professional Officer Program (sponsored by CIDA), the province of Quebec's Quebec Sans Frontieres; and
• a number of volunteer and non-profit organizations, such as Association Internationale des Etudiants en Sciences Economiques et Commerciales (AIESEC, or International Association of Students in Economics and Business Management in English), and the Canadian University Service Overseas (CUSO) , were mentioned as providing business exchanges to youth.

Typically, interviewees provided little information about the programs mentioned above. When details were made known, the Objectives and organizational structure of these programs proved different from those of YPI. In particular, the stipend offered to participants in these programs (if one exists at all) also tended to be lower than that provided by YPI.

The YPI partnership approach, which is fairly unique to this Program, was said to contribute to its success. The partnership involves DFAIT collaborating with Implementing Organizations which in turn collaborate with Host Organizations to set up placements for Canadian youth overseas. According to key informants, by drawing on the resources and connections of each partner, this collaborative approach was said to facilitate funding participants, finding participant placements, and developing networking opportunities. According to most key informants, providing stipends to participants, which is not common outside of YES internship programs, has several advantages: paid participants can stay longer and therefore make more meaningful contributions to their Host Organizations, and the allowance enables participants to cover their international transportation costs (which typically limit access in unpaid programs due to affordability issues).

Further, the YPI criterion that participants must have completed a post-secondary degree or diploma means that participants have more skills than undergraduate students typically placed through other internship programs (the majority of which do not have this requirement)

b) Leveraging of Funding

Contributions leverage from partner organizations varies widely. Based on a sample of 20 YPI projects reviewed from 2003-2004 and 2004-2005, the mean contribution by Implementing Organizations to YPI projects was approximately $39,000, ranging from $2,100 to $165,000 per Implementing Organization and representing 0.8-47.6 per cent of the total funding for the project from all sources. The mean Host Organization contribution was approximately $92,000, ranging from $0 to $235,000 per project, and representing 0-68 per cent of total funding from all sources.

The YPI reports to YES further indicate a great deal of additional contributions beyond what the Federal Government contributes. The 2004-05 report indicates that Implementing Organizations provided a total of about $393,000 in cash contributions and about $1.1 million in in-kind contributions to the Program; Host Organizations contributed a total of about $540,000 and $3.3 million, respectively. The figures further show that the federal government provided $300,000 in in-kind contributions to cover the cost of placement assessments conducted by DFAIT and the cots of the project selection committee.

YPI staff and Implementing Organization representatives interviewed agreed that the amount of funding and in-kind contributions that DFAIT is able to leverage from Implementing Organizations and Host Organizations varies widely depending on the resources of the organizations. Typically, Implementing Organizations in the private sector and Host Organizations in wealthier countries and/or in the private sector are able to provide more funding. Implementing Organization funding is most often in-kind, whereas Host Organizations may supplement participant stipends from YPI. Some Implementing Organizations will work with only Host Organizations that provide both cash and in-kind support or cover airfare. The majority of Host Organizations help participants find lodging, offer advice on local money management and/or top-up participant stipends. Top-ups are most commonly provided by trade-related private-sector Host Organizations, which can typically afford to supplement participant allowances, unlike NGOs. It should also be noted that in selecting Implementing Organizations and their projects for YPI funding, the YPI Implementing Organization selection committee considers top-ups as a criterion for funding, though the substance of the placement itself has top priority.

Evidence suggests a wide range of capacity among partner organizations. The review of the files of 20 CDS projects from the 2003-04 and 2004-05 fiscal years indicates that Host Organizations' financial and in-kind contributions to projects ranged from zero dollars to $235,000 (or 0-68 per cent of the total contributions for a project), while Implementation Organizations' contributions ranged from $2,100 to $165,000 (1-48 per cent of total contributions). However, financial data on contributions by type of organization, which would have demonstrated whether or not there was variation in contributions by type of organization, are not available. Moreover, data on the number of participants by type of Implementing Organization revealed little variation in either the average of or the range (minimum-maximum) in the number of participants per organization.

Based on a comparison of YPI to two government programs with international placement components, the degree of leveraging was higher in one of the programs compared to YPI. The leveraging obtained by CIDA for its International Youth Internship Program (IYIP⁽²³⁾) was much greater than for YPI. Non-government contributions are 91 per cent of government funding for IYIP compared to YPI's 75 per cent. There was no indication of leveraging for Netcorps⁽²⁴⁾ (though it is possible this was not an issue in this evaluation, but there may well have been non-government contributions in this program).

2.4 Program Design and Delivery

a) Equity Group Representation

Program data indicate that the representation of different equity groups matches fairly closely targeted proportions as well as population proportions, apart from women. In every year, women exceed the 50 per cent target by a fair amount, particularly in 2005-06 (74 per cent) (Table 2.1A, with actual numbers being shown in Table 2.1B). The proportion of women among participants also exceeds the population proportion (50 versus 59 per cent). Even comparing the proportion of women among YPI participants to the proportion among those granted a post-secondary certificate of some kind over the 1997-2003 period (59 per cent) would suggest that women are over-represented among YPI participants. The representation of persons with disabilities is somewhat lower than the targeted proportion and than their share in the Canadian population (one versus three-four per cent). This is so despite the efforts GHY has made to encourage organizations to hire people with disabilities, including having sections on employment equity (including people with disabilities) in the proposal guidelines and in the project co-ordinator's manual, and sponsoring a seminar by the Canadian National Institute for the Blind on hiring visually impaired people. The Canadian National Institute for the Blind has been a partner since 1997, sending out many visually-impaired participants. One difficulty pointed out in this regard is that Implementing Organizations dedicated to persons with disabilities do not always participate in the Program.

Source: Program data - reports submitted by Program to YES; and Statistics Canada data:
University degrees, diplomas and certificates granted by sex (averaged 1999-2003);
Visible minority population by age group (age 15-24);
Participation and Activity Limitation Survey: a profile of disability in Canada (age 15-24); and
Population reporting Aboriginal identity by age group (age 15-24).

Source: Program data - reports submitted by Program to YES

Respondents raised some issues when asked to discuss the success of measures adopted by YPI to ensure the appropriate representation of designated target groups, and challenges encountered in doing so, as the following indicates:

• Women: YPI staff and managers reported that the Program has, in fact, been "too" successful with women, which is supported by the results above. Implementing Organizations and Host Organizations appear to support this contention, with many saying that the majority of their participants have been female, and that there is perhaps a need to work on attracting more male participants. The staff and management interviewees suggested that this over-representation of females simply reflects current university demographics in Arts Faculties; and certain Implementing Organization and Host Organization interviewees agreed. The results presented above in Table 2.1A indicate women are somewhat over-represented among YPI participants.
• Aboriginal persons: Staff and management stated that GHY had been successful with respect to Aboriginal persons, that several Implementing Organizations specifically target such persons, and that they have worked some organizations to increase capacity to accommodate Aboriginal persons. Some Implementing Organizations identified some challenges associated with recruiting such participants. Finding Aboriginal youth with post-secondary education and in the age range targeted by the Program was considered a challenge, as a representative of one Implementing Organization pointed out that Aboriginal persons often enter post-secondary institutions as mature students. In fact, an evaluation conducted by EKOS Research Associates for the Natural Sciences and Engineering Research Council (NSERC) of the University Faculty Award (UFA) Program found strong evidence to support this view. On the other hand, the results presented above would suggest that the Program has done well to bring Aboriginal persons, whose proportion of participants exceeds that of the total Canadian population (six versus four per cent nationally). Another problem identified by an Implementing Organization representative is that additional language requirements sometimes imposed by organizations tend to disqualify Aboriginal persons; they typically speak English, French and their own language and are unlikely to have learned any others (such as Spanish).
• Visible minorities: These groups appear to be reasonably well-represented, as per the responses of Implementing Organizations and Host Organizations in the key informant interviews. Around half of the representatives of these organizations said that they were aware of youth from visible minorities participating in the Program. One Implementing Organization representative suggested that some internationally-oriented organizations may have a natural bias towards minority groups due to needs related to language competencies. Staff and managers feel that accurately measuring the representation of visible minorities can be somewhat tricky because they are self-identified.
• Persons with disabilities: YPI staff and managers said that Implementing Organizations are asked what they are doing to recruit those with disabilities, indicating efforts are made to include these persons (as described above). Some Implementing Organizations identified challenges associated with accepting participants with disabilities - for example, accessibility can be a problem in some countries and it is sometimes challenging to take these participants out of their support network.

b) Effectiveness of Safety and Security Measures

All Implementing Organizations submitting proposals for Youth Employment Strategy funding must describe a safety and security strategy (including a contingency plan) that they will put into practice when they receive funding. The strategy includes the following elements:

• Every proposed placement with a foreign Host Organization is reviewed by Canadian Missions (Embassies, etc.) in order to help ensure the safety and security of participants. The DFAIT Proposed Placement Assessment Form (PPAF) is used for this purpose. The Implementing Organization provides the contact details of the proposed placement to GHY in DFAIT, which then forwards the PPAF to the relevant Mission for review. (The Geographic Division and/or the Consular Division within DFAIT may also comment.) If a problem is identified, GHY contacts the Implementing Organization to address the problem or to request an alternate placement. It should be noted that Implementing Organizations are aware that no YPI placements can take place in a country for which there is a DFAIT travel warning;
• Each participant is required to register on-line in the ROCA (Registry of Canadians Abroad) on the DFAIT Consular Affairs website, which facilitates contact and assistance in case of emergency (e.g., conflict, natural disaster);
• During the pre-departure orientation, a briefing on Safety and Security must be provided by the Implementing Organizations to participants, based on a PowerPoint presentation deck and speakers notes on health, safety and security issues, which were developed and provided by YPI; and
• Country information packages are sent to participants, all of whom are encouraged to check DFAIT's travel report site.

According to YPI staff and managers, the safety and security of participants are of paramount importance. YPI has a staff member dedicated to safety and security. Travel reports are read and participants are required to sign waivers. In countries where safety is of particular concern (South Africa, for example), Implementing Organization funding top-ups may provide additional measures, such as ensuring that participants stay in safer accommodations.

Overall, positive impressions of YPI safety and security measures are held by Implementing Organizations, Host Organizations and overseas posts. There was a sense among some Implementing Organizations that safety and security measures have improved over the years. Some safety measures cited by Implementing Organizations and Host Organizations as being particularly useful include orientations and briefings at Host Organizations, call network systems, and on-line forums. Where necessary, these organizations recognized their own vital role in safety and security, and used funds for such practices as paying for housing for the first month, assistance with finding safe housing, and driving services. Despite the strong emphasis on the safety of participants, a few incidents were mentioned by interviewees. For example, an evacuation from the Ivory Coast was not handled as well as it could have been by the consular service; in another case someone was attacked, and in another, someone's bag was stolen. In the latter two cases, the interviewees reflected that "you can't predict everything in life" and "things happen here (in Canada), too." A review of a site visit report indicated that security concerns sometimes arise. On the other hand, praise was given for how situations such as the 2005 Indian Ocean Tsunami and an earthquake in India were handled, with respect to ensuring the safety of affected YPI participants.

The YPI Passport document, now on-line, lays out the YPI security measures. Available to successful Implementing Organization applicants, it is divided into three sections: essentials, YPI forms, and useful resources and other information. The document describes Implementing Organizations' responsibilities with respect to safety and security. The 2005 Passport illustrates the safety measures taken on by the Department. It enumerates safety topics that should be covered in the pre-departure briefings, including: the detailed process of advance assessment of proposed placements, evacuation procedures, natural disaster procedures, travel waiver conditions, registration of Canadians abroad (ROCA) procedures, travel documentation information, DFAIT information and materials available to participants, participant travel procedures, how to deal with trouble, and emergency contact information. Sexual harassment and health and safety should be addressed with participants prior to departure.

Responsibilities of Implementing Organizations for participant safety are also detailed in the Contribution Agreement: "The Implementing Organization shall make all reasonable efforts to ensure that the placements are conducted in a safe and supervised environment and that participants have been informed of the risks involved in an International placement in the country/region to which they have been sent." The Agreement also states that "The Implementing Organization shall make all reasonable efforts to ensure the placement is conducted in a safe and supervised environment, free from harassment."

The following security and safety issues emerged from a review of Roundtable proceedings among participating Implementing Organizations. First, as noted by a presenter at the YPI Roundtable in Halifax in 2004:

"Giving the Young Professional details from the DFAIT Travel Advisories is not enough. Some of the YPs will be sophisticated travelers and think that they are "street smart". No one can afford to be blasé about living and working in a foreign environment. Today's safe city/region/or country can be tomorrow's conflict zone. Our own experience with Cote d'Ivoire in 2002 and Kosovo in March 2004,is a case in point. The security situation virtually changed from one of the safest countries in Africa, and a stabilized one in the Balkans, to two of the most unsafe - in one week! My advice to you and your YPs is never to be complacent. Keep a daily watching brief on the events in your host country."⁽²⁵)

Second, a roundtable presentation held in 2004 in Vancouver addressed issues around gender and safety.⁽²⁶⁾ The presentation focused on working in countries that may perceive gender differences differently than in Canada. Some recommendations were made, as follows:

• Mapping out perceptions and expectations and comparing these to possible differences in other cultural contexts;
• Identifying key areas that one will need to respect and possibly alter in one's behaviour in order to be safe and accepted, including appropriate and respectful body language, dress, not venturing off alone, and not being careless in crowded situations; and
• Realizing that, as an intern, one is in the host country to learn and support the host agency.

c) Potential Utility of Skills Assessment and Virtual Campus Tools

GHY has developed an international employability skills measurement tool for participants, called the Skills Self-Assessment Tool. The YPI Passport states that the tool is intended to be used as a self-growth tool to aid participants in developing and implementing personal skill development plans. The tool was designed to enable participants to determine which international work skills they possess and which ones they need to improve or develop. The skills that have been identified, through independent research, as important to success in an internationally-focussed career include: interpersonal, cross-cultural, adaptation, communication, organizational awareness, research and analysis, problem solving, logistical, computer and management.

Participants complete the skills assessment tool before their placement and, together with the Coordinator, use the results to design a personal skills development plan that sets out the skills they have identified as the most important for them to improve or develop in the context of their career plan. The skills development plan also identifies how the participant plans to meet his or her learning needs (through, for example, pre-departure activities, the placement itself, the Virtual Campus, or other formal or informal learning mechanisms). Participants are also encouraged to use the skills assessment tool during and after the placement in order to measure their progress with regard to skills attainment.

The other tool, the Virtual Campus, comprises a series of one-hour independent learning modules, accessible via the Internet, CD-ROM or in print format. This is one of several ways participants can work on developing the skills they have identified as important using the skills assessment tool. Approximately 32 learning modules are available in the Virtual Campus in areas that correspond to the skills in the assessment tool. There is also a Debriefing module designed to provide YPI participants with information on re-entry shock and job-search skills and strategies, as well as knowledge-based modules on human rights, gender mainstreaming, sustainable development and the United Nations.

These assessment and learning tools are fairly new and therefore measurement of their impact to date cannot be made. The skills assessment tool was introduced in 2003. However; in the first year (2003-04), it was completed in a spreadsheet format and in the following year (2004-05), the on-line interactive version was launched, but did not work well owing to problems encountered with the Internet Service Provider (ISP). Since changing the ISP during the past year (2005-06), the tool has worked well on-line. When used by enough participants at a given organization, meaningful reports may be produced for the organization. The Virtual Campus is also fairly new (introduced in 2003), which makes it difficult to assess at this time.

There is some, but limited, evaluative data on the perceived effectiveness of the Virtual Campus. Following completion of a training module, participants are asked (not required) to complete an evaluation form⁽²⁷⁾ about it. Questions asked include attainment of personal learning Objectives, degree to which knowledge or skills have been increased as a result of the module, the effectiveness of the module, and the helpfulness of the on-line resources. However, since the Virtual Campus's Introduction in 2003, just 35 evaluation forms have been completed by participants. The fact that those who completed forms represent only a very small proportion of those who participated in the training modules⁽²⁸⁾ means the results cannot be considered representative of the views of all users.

Bearing in mind the small number of completed evaluations, participants' views of Virtual Campus training modules are generally positive, with only a few minor negative opinions. The main negative view was that the modules are too short and do not provide enough "in-depth" coverage of the topics. However, it should be noted that participants are told at the start that the training module is not meant to provide a comprehensive summary of this topic. Rather, it aims to provide a useful Introduction to the subject, with participants being encouraged to use the links located throughout and at the end of the module to explore additional points of interest. Also, many of the improvements suggested in the first evaluations have since been implemented (for instance, more quizzes, printable versions, more suggestions for handling stress, etc) and complaints about problems navigating the modules are being addressed through the provision of printable PDF documents.

Implementing Organization representatives have mixed opinions of the potential utility of the Skills Self-Assessment and Virtual Campus tools. Many interviewees feel that these tools are well-designed and "look good", but are not always practical. The two main issues identified by the respondents were Internet access (not everyone has high-speed access at their placements) and time (many participants do not want to spend two hours sitting at a Virtual Campus learning module at night after work). One Implementing Organization representative reported that the organization had adapted the skills assessment tool by asking participants to complete it on paper and then working with them face-to-face. Many Implementing Organization representatives interviewed said that they simply have not kept track of how many participants use these tools.

Finally, as an indicator of the utility of the YPI skills assessment tool, representatives of other YES internship programs have adopted and adapted this tool for their own programs such as Canadian Heritage (Young Canada Works) and CIDA (International Youth Internship Program).

d) Adequacy of Resources

One fairly contentious issue in the delivery of YPI is the amount of money allocated to cover costs associated with each placement. Implementing Organizations receive $15,000 for each placement - $3,000 for administrative costs and $12,000 towards the participant's expenses (living expenses, transportation). Implementing Organization interviewees did not have major issues with the amount provided to cover administrative costs, but had some concerns with the $12,000 for participants. Some highlights of these concerns follow:

• The funding amount has not changed since the inception of the Program almost a decade ago while costs for many things have gone up (it should be noted, however, the value of the Canadian dollar has appreciated considerably in recent years against most currencies, which has to some extent compensated for inflation in host countries);
• The cost of living may vary drastically between countries, but often even in traditionally "inexpensive" places such as New Delhi, people have "caught on" about visitors from other countries and now charge three times the price to people who are not locals;
• Plane tickets and insurance are major expenses that consume a lot of the funds, and in some places cost-saving measures such as inflexible tickets are just not a good idea;
• Medication costs in developing countries are considerable -- anti-malaria drugs cost $5 a day, and some vaccinations cost as much as $900 (though this is something that should have been taken care of in Canada prior to the individual's departure for the foreign country); and
• Finally, some participants are forced to cover many expenses themselves and that means some worthy candidates simply cannot participate because they cannot afford to do so.

Program staff and management are aware of these kinds of difficulties to a certain extent. One commented, "In some instances this (amount) may seem inadequate". However, they made the following observations counter-acting or at least softening these difficulties:

• In Washington, DC (where the cost of living is high), all participants mentioned money as a problem. However, in a site visit, it appeared to be a real issue for only a few participants;
• In 80 per cent of the countries, participants live comfortably, and may even have higher incomes than their supervisors, and certainly higher than other interns;
• In Western Europe, expenses tend to be higher, but participants are informed up-front of expenses so there should be no surprises and Host Organizations are encouraged to help (e.g., in finding affordable lodging);
• In South Africa, where safety is a concern, the $12,000 is often inadequate - top-ups are needed and provided to enable participants to find safer and thus costlier accommodation;
• Many Host Organizations help participants find lodging, offer advice on local money management (which is an important life-skill), and/or top-up participant funding from YPI (most commonly provided by trade-related Host Organizations, which can afford it, unlike NGOs, though the latter do provide in-kind contributions); and
• It is important to remember that the purpose of participating in YPI is to obtain professional international experience while not slipping into debt -- it is not intended as a source of earnings or savings for participants.

Some interviewees discussed the possibility of leveraging more support from the Host Organizations, but the general feeling is that this is not always possible. As suggested by YPI staff and management, private sector organizations are more apt to provide money from their own pockets, whereas NGOs often have to make do with in-kind support. Also, as indicated above in the Leveraging section, partner organizations contributed a wide range of support to projects, suggesting varying levels of capacity to do so.⁽²⁹⁾

e) Effectiveness of Monitoring and Reporting Activities

The RMAF for YPI outlines required sources for performance measurement. They include the following: administrative files and data; regular performance reports submitted by the Program to YES; and youth input via surveys. These sources have been used in this evaluation.

The YPI Passport document and RMAF lay out in detail the accountability requirements placed on the Program and its partners. The Contribution Agreement with Implementing Organizations provides a reporting template. Participants are required to submit bi-monthly reports to their Implementing Organizations. Implementing Organizations, in turn, are required to report quarterly to DFAIT, and 10 per cent of the payment from DFAIT is withheld pending satisfaction with the organizations' performance. The RMAF indicates the quarterly report structure to be as follows: Statistics; Safety and Security Checklist; Placement Highlights/Success stories; Summary of Activities and Results Achieved; Lessons Learned/Best Practices Contact Information for Participants; and Financial Reports. The review of project files revealed that most organizations do follow this structure. The Final Report included in the files has additional sections on Results and Recommendations.

Additional elements of reporting and accountability include the following:

• Skills assessment -- Participants are asked to complete skills assessment checklists before, during and post placement and use them to assess skill needs and acquisition and in conjunction with the Virtual Campus training tool, which in turn GHY asks participants to complete evaluation forms for (as discussed in a previous section);
• Post-placement surveys -- Participants are asked to complete on-line surveys at six months, 12 months and 24 months after completion of their placement (the results of which were presented earlier in this report). The surveys ask questions about employment, skills developed, contacts maintained, and involvement in international issues. Support from the Implementing Organization before, during and following the placement is asked about in only the six-month follow-up survey; and
• YPI year-end reports to YES -- GHY year-end YES reports to HRSDC include budget information; contributions from other sources; participant information (targeted and actual number of participants in designated equity groups, employment/school status at end of placement, province/territory recruited from location of activities (if in Canada), country of placement and number of participants there); and list of contact information for sponsor (implementing) organizations.

Staff and management interviewed for this evaluation pointed to the following three additional accountability activities of the Program:

• Site visits --These are undertaken to ensure YPI's integrity (accuracy in reporting, identification of issues/concerns). Up to three YPI staff conduct site visits -- different countries/regions are visited each year; at least 60 placements are visited annually; representatives of Host Organizations and embassy staff are interviewed during the visits, and sometimes embassy staff follows up with participants; and local CIDA interns may also be visited.
• Audits -- Five Implementing Organizations are audited annually by the DFAIT audit division and five initial visits are made annually to new organizations. The most recent audit found that, with the exception of the absence of a documented assessment of the project and the recipient, the Program is managed with sound comptrollership practices and complies with legislative and regulatory requirements.⁽³⁰⁾ However, it should be noted that GHY has responded to this finding and now summarizes such information.
• "Help-line" -- Participants know to contact DFAIT if there are any problems during their placements.

Four observations may be made about the Program's accountability requirements, as specified in its RMAF and contribution agreements. First, a review of the Implementing Organizations' reports to YPI indicates that the quality and timeliness of the information and the detail provided in Implementing Organizations' reports vary greatly from organization to organization, despite the presence of a reporting template in the contribution agreements with the Implementing Organizations. The review revealed that supplementary questions were sometimes asked by DFAIT or Implementing Organizations to clarify the content of information submitted in reports, and a few Implementing Organization files contained communiqués from the organization or YPI regarding delays in receiving information. Some Implementing Organizations provide the minimum in response to requirements in their reports to YPI, while others include additional detailed information (such as narratives at the individual participant level; media articles relating to participant activities; copies of letters and e-mails between the Implementing Organization and YPI; suggestions from participants; responses to survey questions on participants' views; and letters of support from Host Organizations).

Second, the report outline for Implementing Organizations indicates that the organizations are required to submit information on lessons learned, best practices and recommendations to DFAIT as part of their regular reporting to the Department under the Program. The Department disseminates information collected from Implementing Organizations through Roundtable discussions groups with participating Implementing Organizations and by presenting information and examples of best practices in the Passport document (evidence on safety issues was presented earlier in this report). Another potential model for sharing best practices is the approach taken by CIDA's IYIP, whereby host organizations' best practices are compiled and disseminated on the IYIP website⁽³¹⁾.

Third, it is noted in the RMAF that Implementing Organizations are asked to collect performance information from its Host Organizations. This is to be done via a questionnaire on such issues as match between participants' skills and Host Organization's needs, whether or not the Host Organization's expectations are being met in particular areas, and Host Organization satisfaction with the process of arranging the placement with the Implementing Organization. Such information would be a good source for performance measurement, both for participants and Host Organizations, particularly if it could be rolled up into a database and analyzed. However, a review of the project files revealed little evidence of completed Host Organization surveys. A potential alternative model would be the host organization survey that is conducted on-line by the CIDA evaluation unit for its international youth internship program (IYIP), which gathers information on organizations' satisfaction with design and delivery aspects of the IYIP, their views on the impact on the intern interns, and whether or not organizations' expectations were met.⁽³²⁾

Fourth, while it is commendable that data are collected on participants in post-placement surveys, there are certain shortcomings of this information. Specifically, the following is noted:

• the skill measures are self-reported, thus introducing certain biases (though it should be noted that this is typical of skill attainment measures, and may be costly and impractical to remedy);
• participants are not asked questions on their skills entering their placement and thus no comprehensive baseline measures of skill levels at the start of placements exist that could have been compared to post-placement levels and thus enable measurement of skill gains for participants;⁽³³⁾ and
• the survey question on participants' entry into international employment is imprecise.
• the response rate is typically less than 60 per cent, raising some concerns about the representativeness of the survey data (despite the fact that participants sign letters of agreement with Implementing Organizations that they will complete these surveys);
• further, the possibility is real that those with more positive (or negative) experiences may be more likely to participate in the survey (response bias);
• survey response declines with time, with response to the 24-month survey being particularly low, thus inhibiting reliable measurement of medium-term employment incidence (attrition is typical in such longitudinal surveys), and certainly precluding measuring long-term impacts, noting as well that, under the terms of the agreement with the YES, GHY is forbidden from maintaining participants' Social Insurance Number, which would have facilitated tracking participants following their placement for purposes of measuring outcomes.

Finally, the qualitative evidence gathered in interviews for this evaluation indicates that Implementing Organizations are generally satisfied with the monitoring and reporting activities associated with YPI. Some respondents praised GHY for being so involved and, as one Implementing Organization representative put it, "actually reading the reports and commenting on them."

A few concerns were raised, however. One respondent feels that there is a lot of reporting to do. Another is concerned that DFAIT may not be getting as much from the reporting activities as they could, because they do not collect data on the number of applicants per year (in fact, they do, as data presented in the first chapter indicate), the level of interest in terms of inquiries about the Program, and its impact on Canada (though the latter would be very difficult to measure).

f) Efficiency and Existence of Operational Constraints

YPI staff and managers feel that, from the perspective of the Government of Canada, there are no operational constraints faced by the Program, pointing to a number of factors contributing to Program efficiency. The fact that there is a division within the Department dedicated to YPI's delivery means that staff are very familiar with it. Also, Program processes are spread out evenly throughout the year allowing for a corresponding even distribution of staff workloads. Also, representatives of Implementing Organizations meet regularly in Roundtables to share experiences and best practices and to address constraints that may arise. Furthermore, the format of contribution agreements for YPI and CIDA's IYIP and is similar, thereby reducing the burden on Implementing Organizations receiving funding from both.⁽³⁴⁾ Program managers also pointed out that the fact that the YPI selection committee is composed of representatives from across the Department contributes to the fairness and objectivity in selecting Implementing Organizations for funding under the Program.

YPI management had one suggestion for improvement: to restore funding removed by the Department (flowing from the federal government's expenditure review), given the demand for the Program (as confirmed in Section 2.1, Relevance). In fact, an even larger program is desired to meet demand, so that more Implementing Organization applications and more participants can be accepted than are now.

Single fiscal year funding was a identified as a constraint by all but one Implementing Organization. The one-year funding cycle is self-imposed by GHY. Interviewees identified a number of key problems associated with this policy. In particular, it was pointed out that Host Organizations frequently do not operate on the federal government's fiscal year. The timing does not work for universities and colleges, as their academic years do not finish until the end of April. That leaves a month to confirm participation on the part of Host Organizations and ensure that proposals from Implementing Organizations are still valid, as well as recruit and select participants (when students are often difficult to reach) and get them to the pre-departure briefing in June. This can be challenging for obtaining visas. The single-year approach is also frustrating for "proven partners", as it is difficult for them to be able to plan for only one year at a time. In addition, having to write a proposal every year takes up valuable resources.

A variety of other constraints were identified but by only one or two individuals each. These include the following:

• Insufficient funding (as discussed earlier), also indicated by the fact that participant applications out-number available placements by 20 to one (Program data);
• The way the Program is funded necessitates working with organizations that can provide much in-kind support (and not all worthy organizations can);
• Age constraints, which prevent many otherwise qualified Aboriginal persons from applying (see subsection on representation of target groups);
• Although the Centre for Intercultural Learning falls under the Departmental mandate, only CIDA interns can access it, while YPI Implementing Organizations have to fund briefings on intercultural issues out of pocket; and
• The Program is constantly under fire from government and being questioned as to its effectiveness.

g) Existence of Alternative Approaches and Suggestions for Improvement

YPI staff and managers believe that the current means of delivering and managing YPI through a dedicated division within DFAIT is the most effective way. They mentioned that Industry Canada's NetCorps program hired a consortium of NGOs to run their program, but in YPI's case, being run internally means that direct connections with embassies and missions can be fully leveraged. The staff and management key informants went on to list a number of other advantages to the current delivery approach:

• A dedicated division is more aware of departmental priorities, and therefore is more likely to convince or encourage Implementing Organizations to support the priorities;
• Similarly, by being located within the Department, the Program feeds directly into policy development of the Department;
• As well, having most branches of the Department represented on its selection committee ensures that projects reflect the needs of the entire Department;
• By being located within the Department, the Program can take advantage of departmental elements, particularly the overseas posts to gather information on the host country;
• The Department would be less likely to be aware of important Program performance details if their involvement was not as hands-on within the Department; and
• Having the Program directly within DFAIT enables it to focus specifically on organizations that share the Department's goals.

Implementing Organizations and overseas posts did not have any major suggestions for alternate delivery models. In fact, some praised YPI's delivery approach; in the words of one respondent: "I like that there's a specific unit I can go to, rather than get lost in the whole department." Another Implementing Organization representative commented that YPI staff knew the history of this organizations' particular project (which had been funded from year to year under the Program), and this in turn contributed to its effectiveness.

A few minor suggestions for improvement were offered by a few key informants and participants each:

• Lengthen the duration of placements -- the private sector is not historically very intern-friendly because they are encouraged to contribute funding for salary and airfare. As such, six months is too short a time for private sector Host Organizations to be making such an investment.
• More coordination between DFAIT and CIDA would make matters easier for participants and Implementing Organizations (though the fact that the organizations' respective internship programs [YPI and YIPI] basically have the same contribution agreement format is useful).
• There could be a special stream for people who want to work at Foreign Affairs, as this is currently not an option for YPI participants. In fact, youth can obtain internships at DFAIT through another federal youth program, Federal Public Sector Youth Internship Program.
• Participants should, but do not always, receive the preparation that Implementing Organizations say they are going to provide before their placements begin.
• It is a Program requirement, as the YPI Passport document indicates, that there be an advanced discussion of a detailed workplan for YPI participants once they arrive at the Host Organizatiion, but not all participants receive this. A few of the case study participants interviewed for this evaluation said they did not really have a well-defined job description, and were used too much as "Jacks of All Trades." As such, participants should have received a clear job description in advance of the placement so that they come understanding what their job will be exactly.
• More concrete assistance in finding good jobs should be provided to participants, who sometimes feel they are left "adrift" following completion of their placements, perhaps through the provision of placement or networking opportunities. However, it should be noted that there is a job search learning module in Virtual Campus.
• The questionnaires that participants are regularly asked to complete by Implementing Organizations contain questions that are seen as repetitive by a few participants in the case studies, and it was suggested that there be more substantive and interesting queries.

Top of page

3.0 Conclusions and Suggestions for Improvement

• 3.1 Conclusions
• 3.2 Suggestions for Improvement

3.1 Conclusions

a) Continued Relevance

There is a continued need for YPI. Despite varied reasons for participating, all persons consulted for this evaluation believe that there is a continued need for YPI. The documentation suggests that this need relates particularly to the skills and employment needs of young people interested in international careers. The Program's focus on underemployment is justified by the fact that over a third of Canadian university students in fields that Program participants typically graduate in have taken jobs for which they are overqualified, as have almost half of all university graduates under 30 years of age. The need for YPI also relates to Canada's outward orientation and its need for people with international skills and for a higher profile in the world through qualified workers. There is also evidence of excess demand for the Program, as the number of organizations and youth applying to participate in it is much greater than funding will allow. Uptake and success for proposals from the private sector are low.

Expectations of the YPI Program are being met. Implementing Organizations and Host Organizations are happy with the results. According to Host Organizations, perceptions of Canadians and Canadian values are being positively affected by the YPI experience. Case study subjects generally feel that, on the whole, they got the experience they desired and sought by participating in the Program.

YPI supports the priorities of DFAIT. This is not surprising, given that the Program was explicitly designed to meet the Department's priorities. Additionally, the YPI's Objectives align with those of Canada's Youth Employment Strategy (YES).

b) Success

YPI provides considerable benefits to the youth who participate in the Program. Post-placement survey results indicate that many youth who participated in YPI possess high levels of targeted skills following completion of their placements, particularly in the areas of inter-cultural effectiveness, adaptability/flexibility and patience/tolerance. A majority of participants report either being employed or returning to school at the conclusion of their placements, and the incidence of employment among former participants rises over the time since completion of their placement. It was not possible to measure the long-term impacts of the Program.

YPI benefits DFAIT in a number of ways. A large number of former YPI participants subsequently joined the Department including Canada's Foreign Service, participants have started NGOs that help to further causes for developing countries related to the priorities of DFAIT, and YPI alumni who work in foreign business effectively become sales people for Canada -- which also addresses the Department's international trade and prosperity Objectives. YPI participants who end up working in foreign organizations also support Canadian interests abroad in terms of promoting Canada's place in the world as well as enhancing trade between the respective countries.

c) Alternatives/Leveraging

The evaluation did not uncover any programs comparable to YPI. Many international programs exist for youth in Canada, but none serves precisely the same purpose or has the same structure as YPI. These programs frequently differed from YPI in terms of such factors as Objectives, organizational structure and the size of stipend offered. One program, the International Youth Internship Program (IYIP) is similar, but it is focused on development issues in developing countries, unlike YPI. The YPI partnership approach in particular is seen as unique and a key contributor to the Program's success.

The amount of financial and in-kind contributions that DFAIT is able to leverage for a YPI project from partners varies widely depending on the resources of the organizations. Implementing Organizations in the private sector and Host Organizations in wealthier countries and/or in the private sector typically are able to provide more funding. Implementing Organization funding is most often in-kind, whereas Host Organizations may supplement participant stipends from YPI. The degree of leveraging in YPI in proportional terms was somewhat lower than in the case of IYIP.

d) Program Design and Delivery

Representation of different equity groups matches fairly closely targeted proportions as well as population proportions, apart from women, who regularly exceed the 50 per cent target. Despite much encouragement by GHY for Implementing Organizations to place participants with disabilities, they are somewhat underrepresented among YPI participants.

Efforts are being made to ensure that the safety and security of participants are of importance, and generally impressions of the safety and security measures implemented by YPI are positive. The YPI Passport document, now on-line, identifies the measures taken on by the Department.

GHY has also developed an international employability skills self-assessment tool for participants, and the Virtual Campus, which comprises a series of one-hour independent learning modules, accessible via the Internet, CD-ROM or in print format. Although the views of the training modules are generally positive, it is still early in their lives and little information on their effectiveness is available, partly due to low utilization of these tools. Furthermore, Implementing Organization representatives have mixed opinions of potential utility of these tools owing mainly to issues of Internet accessibility.

The amount of money allocated to cover costs associated with each placement is a somewhat contentious issue. There are concerns that $12,000 for participants is not sufficient, despite YPI staff opinions to the contrary.

There is a great deal of accountability and performance measurement taking place for the Program. These measures include administrative files and data maintained by GHY; regular performance reports submitted by GHY to YES and reports submitted by Implementing Organizations to GHY; and self-administered regular post-placement participant surveys conducted by GHY -- all of which proved to be good sources of information for this evaluation. Other monitoring activities include site visits in the host country undertaken by GHY to ensure YPI's integrity, and audits of Implementing Organizations conducted by the Department.

However, some gaps were identified in performance measurement. Shortcomings of the post-placement surveys include potential response bias of mainly those with positive (or negative) experiences participating in the survey, and a lack of long-term measures of impacts owing to poor response to the 24-month survey and to the lack of surveys past that point. There is also a lack of quantitative data on impacts on YPI partner organizations (Implementing and Host Organizations) which could be a good additional source of information on participant outcomes. Although most Implementing Organization interviewees indicated that they are satisfied with the monitoring and reporting activities associated with YPI, the quality and timeliness of the information provided in Implementing Organizations' reports vary greatly from one organization to another.

While YPI staff and managers feel that there are no operational constraints faced by the Program, Implementing Organization representatives raised concerns primarily about difficulty in adhering to the Government of Canada's fiscal year and having to apply each year for funding despite a long record of successful applications to the Program.

YPI staff and managers believe that the current means of delivering and managing YPI, i.e., through a dedicated division within DFAIT, is the most effective, and Implementing Organizations and overseas posts did not have any major suggestions for alternate delivery models. In fact, some praised YPI's delivery approach. Only minor suggestions for improvement were made each by one or two individuals.

3.2 Suggestions for Improvement

On the basis of the YPI evaluation findings, the following key recommendations are made to DFAIT:

1. Reconsider the current one-year funding delivery arrangement. The YPI fiscal year does not always correspond to the fiscal year of participating organizations. For example, Host Organizations frequently do not operate on the federal government's fiscal year basis (April 1 to March 31) and universities (as Implementing Organizations) have difficulty recruiting participants in time to allow for a six-month placement before the end of the year. "Proven partners" find it difficult and costly to write a proposal every year. GHY should examine whether funding can be delivered in a manner that is more flexible to Implementing and Host Organizations' needs, and consider the possibility of multi-year funding agreements.
2. Assess the adequacy of participant stipends in various locales. A variety of issues were identified with regard to how much money is given to participants during their placements, regardless of country, ranging from the cost of living in Western European countries to the cost of airline tickets to and medical necessities in more "exotic" locales. Therefore, GHY may want to consider investigating what the relative cost of living is in each placement and whether the $12,000 participant stipend is a realistic amount for their subsistence in each location.
3. Strengthen efforts to collect data on the use/effectiveness of the Skills Self-Assessment and Virtual Campus Tools. Many Implementing Organization representatives interviewed said that they simply have not kept track of how many participants use these tools. Indeed, during the course of this evaluation there was some difficulty getting useful evaluative information on participants' use and views of these tools. A future evaluation of the Program would benefit from having more data on the utility and popularity of these tools, possibly by making stronger efforts to encourage participants to make greater use of the skills assessment tool and the Virtual Campus on-line evaluation form.
4. Ensure collection and synthesis of performance information from partner organizations. While Implementing Organizations do regularly submit performance reports, the information is not synthesized and analyzed. Furthermore, Implementation Organizations are required to obtain performance information from their Host Organizations via a questionnaire addressing such issues as the level of compatibility between participants' skills and Host Organizations' needs, and their satisfaction with the process of arranging the placement with the Implementing Organization. There is, however, limited evidence of Host Organizations completing these questionnaires. If such information was collected on a regular basis and stored in a database, it would be a good source of data on participant and Host Organization performance. GHY should consider the on-line Host Organization survey model used for CIDA's International Youth Internship Program.
5. Remedy shortcomings of post-placement data on participants. There are a number of issues with the post-placement surveys that have inhibited performance measurement, including the potential for response bias; an inability to measure skill gains (because a comprehensive pre-placement measure of skill levels is not available); imprecise questions on entry into the international employment; low response rates particularly in the 24-month survey (despite participants' signing letters of agreement with the Implementing Organizations to participate in the surveys); and a lack of longer-term measure of success. To increase response rates and the ability to measure long-term outcomes, remedies include: having GHY taking measures to maintain up-to-date contact information on participants by sending post cards annually asking participants to update their coordinates and participate in the post-placement surveys; ideally adding surveys beyond the 24-month mark; and strongly urging HRSDC (as administrator of the YES) to permit the collection of participants' Social Insurance Number to facilitate tracking them down for purposes of measuring post-placement outcomes. To increase precision, consider asking questions on skill levels at the start of their placement in the six-month post-placement survey and on the "international-ness" of the jobs they occupy in all post-placement surveys.

Top of page

Appendix A - Management Response

After this evaluation was completed a decision was taken by the Government of Canada to cut funding for this program. In the Government's announcement of its restraint measures on September 25th, YPI was identified as a non-core program. While YPI was beneficial to many young internationally-oriented Canadian professionals, other international internships are available through federal, provincial and private sector programs. DFAIT continues to direct its efforts on the complementary International Youth Program (IYP) in which more than 20,000 young Canadians participate annually. It targets the same age group as YPI but benefits a larger number of young Canadians who gain international work experience through the program. As it is an exchange program, it provides a similar opportunity for thousands of international youth to work in and experience Canada, promoting greater knowledge of Canada internationally and supporting our interests to promote immigration to Canada and address labour market needs.

Recommendation # 1

1. Reconsider the current one-year funding delivery arrangement. The YPI fiscal year does not always correspond to the fiscal year of participating organizations. For example, Host Organizations frequently do not operate on the federal government's fiscal year basis (April 1 to March 31) and universities (as Implementing Organizations) have difficulty recruiting participants in time to allow for a six-month placement before the end of the year. "Proven partners" find it difficult and costly to write a proposal every year. GHY should examine whether funding can be delivered in a manner that is more flexible to Implementing and Host Organizations' needs, and consider the possibility of multi-year funding agreements.

Management Response

Currently agreements can be extended up to 6 months into the next fiscal year. However, regardless of the duration of the agreement there is a requirement to report on funds spent in that fiscal year, at the end of the fiscal year.

Recommendation #2

2. Assess the adequacy of participant stipends in various locales. A variety of issues were identified with regard to how much money is given to participants during their placements, regardless of country, ranging from the cost of living in Western European countries to the cost of airline tickets to and medical necessities in more "exotic" locales. Therefore, GHY may want to consider investigating what the relative cost of living is in each placement and whether the $12,000 participant stipend is a realistic amount for their subsistence in each location.

Management Response

To better manage expectations of participants YPI requires that Implementing Organizations communicate clearly and accurately the cost of living for each host country to the potential participants. YPI requests that Implementing Organizations continue to explore means of in-kind and cash contributions as well as the current practice of sharing funds among participants. It encourages the Implementing Organizations to provide participants with training on budget management.

Recommendation # 3

3. Strengthen efforts to collect data on the use/effectiveness of the Skills Self-Assessment and Virtual Campus Tools. Many Implementing Organization representatives interviewed said that they simply have not kept track of how many participants use these tools. Indeed, during the course of this evaluation there was some difficulty getting useful evaluative information on participants' use and views of these tools. A future evaluation of the Program would benefit from having more data on the utility and popularity of these tools, possibly by making stronger efforts to encourage participants to make greater use of the skills assessment tool and the Virtual Campus on-line evaluation form.

Management Response

Implementing Organizations are required to account for the participant's use of the skills assessment tool and the virtual campus in their final report. YPI analyses the information provided in the final reports on the usage of the skills self-assessment and virtual campus tools and routinely refines and promotes these tools.

Recommendation #4

4. Ensure collection and synthesis of performance information from partner organizations. While Implementing Organizations do regularly submit performance reports, the information is not synthesized and analyzed. Furthermore, Implementation Organizations are required to obtain performance information from their Host Organizations via a questionnaire addressing such issues as the level of compatibility between participants' skills and Host Organizations' needs, and their satisfaction with the process of arranging the placement with the Implementing Organization. There is, however, limited evidence of Host Organizations completing these questionnaires. If such information was collected on a regular basis and stored in a database, it would be a good source of data on participant and Host Organization performance. GHY should consider the on-line Host Organization survey model used for CIDA's International Youth Internship Program.

Management Response

YPI sends an annual report to the YES committee at HRSDC with composite statistics of the makeup of participants concerning gender, employment equity categories and areas of recruitment in Canada. Included in this report are the financial totals regarding all in-kind and cash contributions from partner organizations.

In addition to the statistical report an annual narrative report is prepared with an analysis of the statistics sent annually to HRSDC and of the information drawn from final reports produced by Implementing Organizations.

Recommendation #5

5. Remedy shortcomings of post-placement data on participants. There are a number of issues with the post-placement surveys that have inhibited performance measurement, including the potential for response bias; an inability to measure skill gains (because a comprehensive pre-placement measure of skill levels is not available); imprecise questions on entry into the international employment; low response rates particularly in the 24-month survey (despite participants' signing letters of agreement with the Implementing Organizations to participate in the surveys); and a lack of longer-term measure of success. To increase response rates and the ability to measure long-term outcomes, remedies include: having GHY taking measures to maintain up-to-date contact information on participants by sending post cards annually asking participants to update their coordinates and participate in the post-placement surveys; ideally adding surveys beyond the 24-month mark; and strongly urging HRSDC (as administrator of the YES) to permit the collection of participants' Social Insurance Number to facilitate tracking them down for purposes of measuring post-placement outcomes. To increase precision, consider asking questions on skill levels at the start of their placement in the six-month post-placement survey and on the "international-ness" of the jobs they occupy in all post-placement surveys.

Management Response

There is substantial reporting by the Implementing Organizations regarding skills acquisition by the participants in the quarterly and final reports. Skills development and acquisition are assessed during the placement since it is still fresh in the participants' mind. YPI in collaboration with Implementing Organizations has standardized and categorized the reporting of skills assessment and acquisition in order to refine the quarterly and final reports.

As to the post-placement surveys, they are designed to give an added perspective on the results two years after the completion of the program. YPI in collaboration with Implementing Organizations has increased follow-up to ensure compliance.

As to the collection of the participants' Social Insurance Numbers the Access to Information and Privacy Protection Division (DCP) has provided the following information:

In its response to the parliamentary review of the Access to Information Act and the Privacy Act, the government of Canada committed itself to examining the use of the Social Insurance Number (SIN) within government and putting limits on its use by the federal government. The Government of Canada's policy on Data Matching and Control of the Social Insurance Number was issued in 1989 by Treasury Board Secretariat, and has been incorporated into the current policy on Privacy and Data Protection.

As required by the Policy, government institutions must limit their use of the SIN for administrative purposes to those authorized by statute or regulation and for administering pensions, income tax, health and social programs (as listed in Use of Social Insurance Number Guideline[1] and related legislative uses).

In addition, government institutions must not withhold any right, benefit or privilege nor impose any penalty by reason of an individual's refusal to disclose the SIN to a government institution except for the purposes set out in the guideline or as otherwise authorized by Parliament.

Institutions wishing to initiate a new use of the SIN must have the authorization for use of the SIN included in their legislation. The Legislated uses for the Social Insurance Number as well as authorized uses of SINs may be found on the Treasury Board Secretariat web site.

The Department of Foreign Affairs and International Trade does not have the appropriate authority to collect, use, or disclose Social Insurance Numbers (SIN). Consequently, it is the view of the Access to Information and Privacy Protection Division that GHY and the Young Professional International programme will not be able to collect, use, or disclose SINs to implement the Inspector General's recommendation number #5.

The Access to Information and Privacy Protection Division (DCP) has also found no evidence that the use of the SIN would have provided the program with a more accurate mechanism to locate the recipients. The recommendation has not offered any technique, process or procedures that could identify how collection or use of SIN numbers would have resulted in locating and maintaining closer contact with the recipients. It must be noted that agencies which do have the legislative right to collect SIN numbers, such as HRSDC, neither maintain nor have access to the most up-to-date address and or location of individuals based on their records. Consequently, there is no evidence to show that this recommendation would contribute to the stated objective.

Office of the Inspector General

^* If you require a plug-in or a third-party software to view this file, please visit the alternative formats section of our help page.

Primary navigation

Date Modified:

2012-09-22