Foreign Affairs and International Trade Canada

international.gc.ca

Common menu bar

• Français
• Home
• Contact Us
• Help
• Search
• canada.gc.ca

Breadcrumb

1. Home
2. >
3. About the Department
4. >
5. Accountability

Audit of Desktop Tools (May 2001)

Audit of Desktop Tools

(May 2001)

• 1.0 Executive Summary
• 2.0 Background
• 3.0 Audit Approach
• 4.0 Web Survey - Methodology
• 5.0 Key Findings
• 6.0 Recommendations
• 7.0 Appendices

1.0 Executive Summary

• Audit Division (SIV) and Information Management and Technology Bureau (SXD) identified the requirement to survey DFAIT staff to assess how well DFAIT's standard electronic tools are supporting users in their jobs and the overall level of user satisfaction
• Data was collected through the use of a web survey (representative sample of 644 completed questionnaires) and interviews with staff located at both Headquarters and Missions
• Most DFAIT employees use and are satisfied with the standard software program suite. The most commonly used programs (Outlook Mail, WordPerfect, Netscape, Jetform, Quick View Plus, Outlook Calendar) are used department-wide
• DFAIT supported (standard) software programs were used for their accessibility, ease of use and functionality. Where staff do not use standard software, the primary reason is that they do not need it for their job
• At least four out of ten (43%) of DFAIT staff use non-standard applications, primarily for sharing files with external partners, or where there are no DFAIT supported programs with equivalent functionality
• Information sharing practices are primarily internal to the department, although almost one-third of employees have information sharing requirement with external clients. Email and local shared drives (the "I" drives) are the primary means of sharing information

Recommendations include:

• Re-evaluate current training program to better reflect the needs of the users
• Develop and implement a communication plan to increase staff knowledge of existing guidelines for information sharing and storing information in corporate repositories
• Given the extensive use of the Microsoft Suite, investigate options for reducing licensing costs
• Assess the adequacy of the level of resources devoted to technical support, e.g., through comparisons to other organizations and external benchmarks

Top of Page

2.0 Background

• SIV and SXD identified the requirement for three SXD-related audits in fiscal 2000/2001
• One of the three audits identified was the "Effective Use of Desktop Tools Audit"
• SIV conducted the audit between August and November, 2000
• The purpose of the "Effective Use of Desktop Tools Audit", was to assess how well DFAIT's standard electronic tools are supporting users. The assessment focused on the following broad issues:
  • Which standard programs are users using and how are these programs being used?
  • What non-standard programs are being used, and how are these programs being used?
  • What are the users' information sharing practices?
  • What is the overall satisfaction level of users with DFAIT's standard electronic tools?

3.0 Audit Approach

• Preliminary Survey Phase
  • Preliminary interviews and document review (including review of previous survey findings)
  • Web-survey to accumulate information
  • Analyse survey results and develop preliminary findings

• Detailed Examination Phase
  • Detailed interviews with representatives from four missions
  • Detailed interviews with representatives from three Headquarter's bureaux

4.0 Web Survey - Methodology

• Web Survey of DFAIT Employees
  • Questionnaire was developed and issued
  • Primary data was captured on usage of DFAIT supported (standard) and locally supported (non-standard) software programs
  • Representative sample with 644 completed questionnaires
  • The margin of error is +/- 3.7%, 95% confidence
• Data Collection Period
  • September 28^th to October 4^th, 2000

4.1 Respondent Profile

• 284 respondents were located at Headquarters and 338 at Missions
• 28% of respondents were locally-engaged staff working predominantly in the areas of trade and administration (Appendix I)
• Most respondents were Canada-based staff (72%). Of these, most were officers (53%) (Appendix II)

Top of Page

5.0 Key Findings

5.1 DFAIT Supported (Standard) Software Programs

• Most DFAIT employees use the standard software program suite. Usage of individual applications varies from almost universal (98% for Outlook Mail) to very limited (9% for Desktop Outbound Fax)
• The most commonly used programs (Outlook Mail, WordPerfect, Netscape, Jetform, Quick View Plus, Outlook Calendar) are used department-wide
• Certain applications (Corporate Applications, Conference Rooms, CATS) are used more heavily by HQ staff, while others (Contacts Plus, Outlook Public Folders) are used more by Mission staff
• DFAIT supported (standard) software programs were used for their accessibility, ease of use and functionality
• Where users have not adopted standard applications, the primary reason given is that it is not a requirement for their job, i.e., they don't need it
• Training is an important factor in the adoption and support of DFAIT supported software programs. Not having taken training, rather than availability of training, was the second most important reason for not using the DFAIT standard suite
• Certain applications are not being used as intended, as there is a lack of awareness concerning functionality

Most Commonly Used DFAIT Supported (Standard) Software Programs

• Outlook 98: Mail, WordPerfect and Netscape were the most commonly used DFAIT Supported (standard) Software Programs (refer to Appendix III for proficiency of use)

Use of DFAIT Supported (Standard) Software Program by Location

• A higher percentage of Mission Staff use the Public Folders and Contact Plus
• HQ staff use Corporate Applications, Conference Rooms and CATS more than employees working in Missions

• ^(a) It is likely that these figures are too high, based on the interviews conducted at Missions and HQ. This is probably due to confusion between folders used by individuals to file emails and the shared "Public Folders"

Training Received on DFAIT Supported (Standard) Software Programs

• Mission Staff received more training than HQ staff for all DFAIT Supported Software Programs (refer to Appendix IV for overall results)
• Interviews revealed that availability of training is not the primary reason for insufficient training, rather, it is employees' lack of time to participate in training sessions:
  • "I missed most of the training due to operational priorities"
  • "the tools are there but many people won't take the time to learn them"
• Several interviewees suggested that mandatory training is the solution to ensuring all employees receive the training required

Reasons for not using DFAIT Supported (Standard) Software Programs

• The main reasons for not using DFAIT supported software programs were that it was not a requirement for their job and lack of training on the program
• Comparing reasons by location revealed that staff located in Missions are experiencing slower response times than those located in Headquarters
• Examples of individual feedback on this issue included:
  • "I don't use Public Folders because I don't know where they are"
  • "more and more people have Word at home...find WordPerfect too complicated"
  • "I don't use (standard programs) because I have no use for them..not because of a lack of training"
  • "I don't know about the purpose of some of the standard programs and how they would make my life easier...don't need them for my job"

Use of and Satisfaction with the C4 System

• Approximately one third (32%) of employees surveyed reported that their job required them to use the C4 system
• 58% of Headquarters staff jobs do not require the use of the C4 system, compared to 47% of Mission staff jobs
• Satisfaction with various aspects (i.e., ease of use, technical support, functionality, accessibility and training) of the C4 System is relatively high
• Mission (31%) staff are less satisfied with the amount of training provided on the C4 system than Headquarters staff (46%)

Top of Page

5.2 Locally Supported (Non-Standard) DFAIT Software Programs

Summary of Locally Supported (Non-standard) Software Programs

• At least four out of ten (43%) DFAIT staff use non-standard applications (perhaps over 50%, as one-quarter don't know whether or not they do)
• The main reason for using the non-standard applications is compatibility, e.g., for file sharing, particularly with external partners
• Staff generally feel they have access to the software they need to do their jobs, i.e., if they need non-standard software, they can get it

Usage of Locally Supported (Non-standard) Software Programs

• 43% of DFAIT staff reported that they use locally supported software programs
• Headquarters (49%) staff used locally supported software more than those located in the Missions (37%)
• Internet Explorer and the Microsoft Office Suite are the most commonly used locally supported (non-standard DFAIT) software programs (refer to Appendix V for overall frequency of use)

Training Received on Locally Supported (Non-standard) Software Programs

• Training received on locally supported (non-standard) software programs is proportionately lower than that received on DFAIT supported (standard) software programs
• Of the most commonly used locally supported (non-standard) software programs, between one-quarter and one-third of users received some form of training

Reasons for using Locally Supported (Non-standard) Software Programs

• Compatibility with job requirements and limited functionality with DFAIT supported programs were the main reasons for using locally supported (non-standard) software programs
• Examples of individual feedback on this issue included:
  • "No one is using Corel Presentations...I receive PowerPoint files"
  • "Getting Microsoft Office has helped a lot since many of my contacts use it"

Top of Page

5.3 Information Sharing Practices

Summary of Information Sharing Practices

• Information sharing practices are primarily internal to the Department, although almost one-third of employees have to share information with external clients
• The majority of staff share word processing files, enabled by strong support and implementation of DFAIT standard software programs
• Email and the local shared drive (the "I" drive) are the primary means of sharing information
• As noted above, staff have access to non-standard software when they need to facilitate information sharing with external partners
• Use of the public folders is limited (37%), restricting access across the department
• There are concerns over the organization of the stored information/files
  • document version control
  • organization of folders on the shared drive

Importance of Information Sharing

• 81% of DFAIT employees claim that information sharing is very important in their jobs
• There is significant sharing of information among DFAIT staff at Missions and Headquarters
• 47% of DFAIT employees share files with OGD's and 35% share files with external clients

Information Sharing Methods

• DFAIT employees are most likely to share information by email and shared drives
• Mission staff use public folders more often than Headquarters staff
• Headquarters (40%) staff typically share more presentation files than Mission (25%) staff
• Employees have access to non-standard software when they need it to facilitate information sharing
• Limited usage of the public folders may indicate confusion around the definition/use of tool for information sharing
  • "I don't know what the departmental file classification system is or where to find it:
  • "I don't use the public folders as much as I should because the structures are confusing and it needs to be cleaned up"

Information Sharing Problems

• Most employees (70%) reported having no major problems with sharing file information
• Incompatible programs, technical issues and incompatible versions were the main issues with sharing file information
  • "While Word documents can be converted to WordPerfect, the same is not the case between PowerPoint and Corel Presentations."
  • "Document Version control is a major concern...multiple versions of the same document on diskettes, d:drive, h:drive, and i:drive."

Top of Page

5.4 Overall Satisfaction

Summary of Overall Satisfaction

• DFAIT employees are, in the main, satisfied with the DFAIT standard suite of software programs, although there are pockets of dissatisfaction (15%)
• They are less satisfied with the training and technical support provided
  • training is seen as being available but not sufficiently tailored or flexible
  • the quality of the support is not an issue - the main concern with technical support relates to the timeliness of the response
• Amount of training received by location is not comparable; Mission staff received more training on DFAIT supported software programs than those located at Headquarters

Satisfaction with DFAIT Supported (Standard) Software Programs

• Overall dissatisfaction with standard suite is 15%; 59% are satisfied; 26% are neither satisfied nor dissatisfied
• Employees are most satisfied with Outlook Mail, Outlook Calendar and Netscape and least satisfied with Jetform, Corporate Applications and CATS
• Satisfaction with DFAIT supported software programs by location reveals that Missions staff are less satisfied with Contacts Plus, Corel Presentations 8, Corporate Applications, Desktop Outbound Fax and SIGNET Metaframe/Citrix than Headquarters staff

Overall Satisfaction with Various Aspects of the DFAIT Supported (Standard) Software Programs

• Respondents were most satisfied with the accessibility, ease of use and functionality of the DFAIT supported software programs
• The attributes that have the greatest impact on overall satisfaction with DFAIT supported software programs are: functionality, ease of use and flexibility

Feedback on Technical Support

• Response times to emergencies were of concern. Technical support staff are seen as well qualified, but the function may be under resourced. Missions are less satisfied with technical support than Headquarters staff
  • "Yes, adequate technical support...quick response is limited...usually get response in the day...no express help."
  • "Good technical support...no issues here. We should have mandatory training on software programs. Hardware is not an issue."
  • "Adequate but not timely...hardware support is good."
  • "Please don't change the suite again...just getting used to the current one."
  • "Outlook Computer Based Training (CBT) is excellent...takes about 5 minutes...would like to see more (i.e. Contacts Plus, Public Folders)."

Feedback on Training

Training-related Issues:

• Level of training provided

"Need different training models for different people and to recognize people learn in different ways and at different speeds...need different models for flexibility."

• Timing of course offerings; and

"The training schedule is not conducive to managers' schedules, there is a need for one on one training"

• The inability of course materials to address the direct needs of the users.

"Need to tailor courses to needs of employee (based on their needs, how much information they will use and how they learn)."

"(DFAIT) needs to devise a training tool for the corporation that understands the needs of the users...current learning tools are ineffective, too technical and assume a level of knowledge"

5.5 Effective Use

Effective Use of Desktop Tools

• By pursuing the four key objectives regarding: Standard Tools; Non-Standard Tools; Information Sharing; and Overall Satisfaction, the audit also strove to assess whether staff are using desktop tools effectively.
• Ability to draw conclusions is limited:
  • there are no DFAIT standards or defined levels of expected competency for staff - required competencies would vary widely by position - relevant external benchmarks are not available, and
  • area is subjective - desktop tools are among the tools used by staff - proficiency in their use does not necessarily mean effective use.
• Given these limitations, the audit identified certain items worthy of note.
• While Outlook 98: Mail is the most universally used program, users tend to employ only the basic features, e.g., few use the "message filtering" feature - given the common complaint regarding the excessive amount of email, better use of such a feature could lead to more effective use of the program.
• For information storage and sharing, there are a wide range of practices and tools used. This leads to frustration when attempting to find/share information, reducing overall effectiveness. Adoption of more standard practices department-wide would increase efficiency and effectiveness.
• Outlook 98: Calendar is used extensively, however, few use it as a scheduler - as a result, there is not a "critical mass" to allow effective meeting planning/scheduling. Organizations that have adopted this type of tool find it extremely effective. Adoption in the Missions may have less applicability than HQ, given the relatively high degree of external meetings.

Top of Page

6.0 Recommendations

Recommendation to CFSS:

• Re-evaluate current training program to better reflect the needs of the users. Consider the following:
  • identify which software programs are most critical (e.g., highest usage) and prioritize training course development
  • define minimum competencies required at each level, with emphasis on skills as opposed to software programs
  • develop mandatory training of the DFAIT Standard Software Suite for new hires and define minimum competencies
  • assess training requirements annually and provide "refresher" courses on basic functionality skills

CFSS Response:

• CFSS will study the recommendations in detail and assess what can and cannot be done with its current level of funding and submit an Action Plan by the end of June 2001.

Recommendation to SXD:

• Develop and implement a communication plan to increase staff knowledge of existing guidelines for information sharing and storing information in corporate repositories.

SXD Response:

• Agree. SXD will undertake to establish a communications plan aimed at increasing staff awareness of guidelines for information sharing and the storage of information in various corporate systems. This will be undertaken within the scope of the Information Management projects planned.

Recommendation to SXD:

• Given the extensive use of the Microsoft Suite, investigate options for reducing licensing costs

SXD Response:

• The IMT Steering Committee recently issued a Status Report which recommended that the status quo be maintained for the departmental suite of desktop software over the next few months. SXD anticipates revisiting this activity during 2001-2002 as departmental discussions on this topic continue.

Recommendation to SXD:

• Assess the adequacy of the level of resources devoted to technical support, e.g., through comparisons to other organizations and external benchmarks.

SXD Response:

• SXD accepts this recommendation in principle, however it should be noted that this is an on-going activity within SXD in order to balance service levels and costs on a global scale.

Top of Page

7.0 Appendices

Appendix I

Respondents = 177
Base: All locally-engaged respondents

Appendix II

Respondents = 464
Base: All Canada-based respondents

Appendix III

Self-Assessed Proficiency Level of Respondents

Base: Respondents who use the program

Appendix IV

Percentage of Respondents that Received Training on the Most Commonly Used DFAIT Supported (Standard) Software Programs

Base: Those who use the program

Appendix V

Frequency of Use of Locally-Supported (Non-Standard) Software Programs

• Internet Explorer and Microsoft Word are the locally supported (non-standard DFAIT) software programs used most frequently
• Headquarters staff use Microsoft Photo Editor and Microsoft Access more frequently than Mission staff

Office of the Inspector General

Primary navigation

Date Modified:

2008-10-24