Glossary of results-based management terms

Source: Results-Based Management for International Assistance Programming: How to Guide.

Accountability

The obligation to demonstrate that responsibility is being taken both for the means used and the results achieved in light of agreed expectations.^{Footnote 1} While no one organization or project is entirely responsible for the achievement of outcomes—especially at higher levels in the results chain—the implementer is responsible for designing a project with achievable expected outcomes, and demonstrating that it is Managing for Results, i.e. that:

• expected outcome and output indicators are established,
• monitoring, including data collection on output and outcome indicators is regularly undertaken,
• management decisions are informed by the data collected and its assessment,
• corrective action is undertaken so the expected outcomes can be achieved, and
• reports on outcomes achieved are supported by evidence.

Activities

Actions taken or work performed through which inputs are mobilized to produce outputs.

In Global Affairs Canada-funded projects, activities are the direct actions taken or work performed by project implementers.

Actual data

Actual data is:

• collected on each indicator as per the collection frequency identified in the performance measurement framework during implementation and documented in various reports and data systems
• used for analysis to assess progress on or towards expected outcomes, in comparison to baseline data and targets
• used as evidence of progress on or towards or on the achievement of an expected outcome in the narrative of performance reports

Assumptions

Assumptions are the conscious and unconscious beliefs we each have about how the world works. From the perspective of the design team, assumptions constitute beliefs (validated or otherwise) about existing conditions that may affect the achievement of outcomes and about why each level will lead to the next. In the context of the theory of change and logic model, assumptions are the necessary conditions that must exist if the relationships in the theory of change are to behave as expected. Accordingly, care should be taken to make explicit the important assumptions upon which the internal logic of the theory of changeis based.

Arrows between the levels represent assumptions (explained in the theory of change narrative) about why the outputs or outcomes from one level should lead or contribute to the changes at the next level, and about existing conditions, including risks, which may affect the achievement of the outcomes.

Attribution

The extent to which a reasonable causal connection can be made between a specific outcome and the activities and outputs of a government policy, program or initiative.^{Footnote 2}

Note: In a multi-donor context, there is an accepted understanding that Global Affairs Canada does not work alone in the achievement of results, and that accountability for tracking progress on expected results is shared by partners and other stakeholders. Areas of work can span diverse sectors, encompass various actors (from the community level through to the international level), and reach across many countries.

Baseline (Data)

Baseline data provides a specific value for an indicator at the outset of a project or program. Baseline data is collected at one point in time, and is used as a point of reference against which progress on the achievement of outcomes will be measured or assessed.

Beneficiary

The set of individuals that experience the change of state, condition or well-being at the ultimate outcome level of a logic model. In its international assistance programming, Global Affairs Canada-funded implementers usually work through intermediaries to help achieve changes for beneficiaries. Global Affairs Canada implementers may also work directly with beneficiaries. In this case, beneficiaries may, like intermediaries, also experience changes in capacity (immediate outcome), and changes in behaviour, practices or performance (intermediate outcome).

See also Intermediary and Stakeholder.

Data collection methods

Data collection methods represent how data on indicators are collected. Choosing a data collection method depends on the type of indicator and the purpose of the information being gathered. Data collection methods can be informal and less structured, or more formal and more structured. Different methods involve “trade-offs with respect to cost, precision, credibility and timeliness.”^{Footnote 3}

Data sources

Data sources are the individuals, organizations or documents from which data about your indicators will be obtained. The implementer will need to identify data sources for indicators. Data sources can be primary or secondary.

• Primary data is collected directly by the implementer at the source.
• Secondary data is data that has been collected and recorded by another person or organization, sometimes for altogether different purposes.

Development results

Development results are a sub-set of results of the Global Affairs Canada’s international assistance results (or outcomes) focused specifically on producing tangible improvements in the lives of the poor and vulnerable. In the Department’s results chain for international assistance programming, these would be changes described at the immediate, intermediate and the ultimate outcome levels.

See also Outcomes or Results, Immediate Outcomes, Intermediate Outcomes, Ultimate Outcomes.

Donors

Global Affairs Canada or another donor organization that provides financial, technical and other types of support to a project.

See also Stakeholder

Evaluation

“Evaluation is the systematic and objective assessment of an on-going or completed project [or part of], programme or policy, its design, implementation and results”.^{Footnote 4}  “In the development context, evaluation refers to the process of determining the worth or significance of a development [initiative].”^{Footnote 5}

Expected outcome or results

An outcome that a program, policy or initiative is designed to achieve.^{Footnote 6}

Ex-post evaluation

“Evaluation of a ... [initiative] after it has been completed. Note: It may be undertaken directly after or long after completion. The intention is to identify the factors of success or failure, to assess the sustainability of results and impacts, and to draw conclusions that may inform other [initiatives]”.^{Footnote 7}

Immediate outcome

See Immediate Outcome below under outcome.

Implementer

Private firm, non-governmental organization, multilateral organization, educational institution, provincial or federal government department or any other organization selected by Global Affairs Canada to implement a project in a partner country. Depending on the context, an implementer may be referred to as an implementing organization, executing agency, partner or recipient.

See also Stakeholder

Indicator (performance)

An indicator, also known as a performance indicator, is a means of measuring actual outcomes and outputs. It can be qualitative or quantitative, and is composed of a unit of measure, a unit of analysis and a context. Indicators are neutral; they neither indicate a direction of change, nor embed a target.

Indicators (qualitative)

Qualitative indicators capture experiential information, such as the quality of something, or beneficiaries’ perception of their situation. They can help measure the presence or absence of specific conditions, or an individual or group’s perception of how a service compares with established standards. Qualitative indicators can capture contextual information about situations, events and practices. For example, “level of confidence (1-4 scale) of farmers (f/m) in the security of roads leading to local market” or “%/total individuals (f/m) who felt that they were completely or mostly able to participate in democratic management bodies”.

Indicators (quantitative)

Quantitative indicators are used to measure quantities or amounts. For example; “# of human rights violations", "ratio of women-to-men in decision-making positions in the government", or "%/total of women-owned businesses represented in trade fairs".

Indicators (leading, lagging and coincident)

A leading indicator signals a future event. A lagging indicator is one that follows an event. A coincident indicator occurs at about the same time as the conditions it signifies.^{Footnote 8}

We generally use indicators to measure progress on outcomes in the logic model. Coincident indicators are generally preferred as they offer the most concrete evidence of changes described in the expected outcomes of a logic model.

Sometimes, however, you may also want to measure the assumptions articulated in the theory of change narrative represented by the arrows in your logic model. In this case you can use “leading” indicators to measure things preceding the change or “lagging” indicators to measure things that follow the change. Data on these indicators can validate these assumptions.

Inputs

The financial, human, material and information resources used to produce outputs through activities in order to accomplish outcomes.

Intermediary

Individual, group, institution or government, that is not the ultimate beneficiary of the project, but that will experience a change in capacity (immediate outcome) and a change in behaviour, practices or performance (intermediate outcome) which will enable them to contribute to the achievement of a sustainable change of state (ultimate outcome) of the beneficiaries. Intermediaries are often mandate holders or duty bearers that are responsible for providing services to the ultimate beneficiaries. They are the entities that implementers work with directly.

See also Beneficiary and Stakeholder

Intermediate outcome

See Intermediate Outcome below under outcome

Logical framework analysis

(Replaced by the logic model, performance measurement framework and risk register in fall 2008)

The logical framework analysis is a planning and communications tool that describes the intent of an investment and presents expected results, indicators, risks and mitigating strategies.

Note: The logical framework analysis is a Results-Based Management tool that was used by Global Affairs Canada until 2008. The logic model, the performance measurement framework and the risk register have since replaced the logical framework analysis. However, logical framework analysis is still used by others in the international development community and officers may encounter it when working with partners tools (Multilateral, sector-wide approaches, programs-based approaches, etc.)

Logic model

Like a roadmap or a blueprint, a logic model is a visual depiction of the main elements of a theory of change for a specific project or program, reflecting the series of changes that are critical to achieving project success. It depicts the logical connections between the planned outputs and the expected outcomes (immediate, intermediate and ultimate) that the project aims to achieve or contribute to. The logic model forms a pyramid shape with multiple complementary pathways branching off below one ultimate outcome level.

The logic model is used as both a planning and design tool during the development of a project or program, and a management tool during project or program implementation.

As of 2016, Global Affairs Canada project level logic model contains the following levels: ultimate, intermediate and immediate outcomes and outputs.

See also Results Chain

Outcome or result

Results are the same as outcomes. An outcome is a describable or measurable change that is derived from an initiative's outputs or lower-level outcomes. Outcomes are qualified as immediate, intermediate, or ultimate; outputs contribute to immediate outcomes; immediate outcomes contribute to intermediate outcomes; and intermediate outcomes contribute to ultimate outcomes. Outcomes are not entirely within the control of a single organization, policy, program or project; instead, they are within the organization's area of influence. In the context of development, these are also referred to as development results.

Global Affairs Canada uses the terms results and outcomes interchangeably throughout its different documents.

Three types of outcomes related to the logic model are defined as:

Immediate outcome

A change that is expected to occur once one or more outputs have been provided or delivered by the implementer. In terms of time frame and level, these are short-term outcomes, and are usually changes in capacity, such as an increase in knowledge, awareness, skills or abilities, or access* to... among intermediaries and/or beneficiaries.

*Changes in access can fall at either the immediate or intermediate outcome level, depending on the context of the project and its theory of change.

Immediate outcomes articulate the changes in capacity that intermediaries and/or beneficiaries should experience during the life of a project. For example: "Improved knowledge of sustainable agricultural-production practices among women smallholder farmers in village X, of country Y", or "Improved business skills of urban women and youth in city Y of country X.

Intermediate outcome

A change that is expected to logically occur once one or more immediate outcomes have been achieved. In terms of time frame and level, these are medium-term outcomes that are usually achieved by the end of a project/program, and are usually changes in behaviour, practice or performance among intermediaries and/or beneficiaries.

Intermediate outcomes articulate the changes in behaviour, practice or performance that intermediaries and/or beneficiaries should experience by the end of a project. For example, "Increased equitable access to safe, quality education for girls and boys in crisis-affected province Y of country X", or  "Enhanced^{Footnote 9}protection of the rights of minorities by government X in country X".

Ultimate outcome

The highest-level change to which an organization, policy, program, or project contributes through the achievement of one or more intermediate outcomes. The ultimate outcome usually represents the raison d'être of an organization, policy, program, or project, and it takes the form of a sustainable change of state among beneficiaries.

The ultimate outcome represents the “why” of a project and should describe the changes in state, condition or well-being that a project’s ultimate beneficiaries should experience. These should not be confused with changes in surrounding circumstances, such as increased economic growth […]. In the context of international assistance programming, an ultimate outcome should instead reflect changes in the lives of women, men, girls and boys in the partner country, such as “Increased empowerment of women in village Y of country X”, or “Improved equitable health of girls and boys under age five in rural areas of region X”.

Outputs

Direct products or services stemming from the activities of an organization, policy, program or project.

In Global Affairs Canada’s results chain for international assistance programming, outputs are the direct products or services stemming from the activities of an implementer.

Outputs and activities matrix

The outputs and activities matrix is a companion to the logic model and the theory of change narrative. Together, they capture the project’s theory of change along the Global Affairs Canada results chain, from the ultimate outcome to the activities and, if the outputs and activities matrix is used to develop an outcome or output-based budget, to inputs.

The outputs and activities matrix breaks down the outputs into the activities required to produce them. The outputs and activities matrix is presented as a table. It repeats the immediate outcome and output levels from the logic model in order to facilitate cross-referencing between both documents. This also allows the reader to follow the logic of the results chain from the activities to the immediate outcome level.

Performance management

Performance Management refers to the various business processes associated with the performance functions of the department and its programs. It includes results-based management, integrated risk management, performance reporting, evaluation, and audits.

Performance measure

See Indicator (Performance)

Performance measurement

“The process and systems of selection, development and on-going use of performance measures [indicators] to guide decision-making.”

See also Results-based Monitoring and Indicator (Performance)

Performance measurement framework

The performance measurement framework is the Results-Based Management tool used to systematically plan the collection of relevant data over the lifetime of the project, in order to assess and demonstrate progress made in achieving expected results. The performance measurement framework is the “skeleton” of the monitoring plan: it documents the major elements of the monitoring system in order to ensure regular collection of actual data on the performance measurement framework indicators. The performance measurement framework contains all of the indicators used to measures progress on the achievement of the project’s outcomes and outputs. In addition, it specifies who is responsible for collecting data on the indicator, from what source, at what frequency and with what method. It also includes the baseline data and target for each indicator.

Performance reporting

The process of communicating evidence-based performance information, including progress on or toward the expected outputs and outcomes: comparing what you expected to achieve with what you have actually achieved, and explaining any variation between the two. This evidence should include the data collected on the performance indicators identified in the performance measurement framework (or equivalent) to measure these outputs and outcomes. Performance reporting supports decision-making, accountability, transparency and managing for results.^{Footnote 10}

See also Progress on and Progress towards

Progress on and progress toward

When reporting on outcomes, you can speak about progress “on” or “towards” the achievement of that outcome. This difference allows you to report on progress “towards” an outcome early in the life of the project even when there has not been a significant change in the value of the indicators for that outcome.

• Progress onis defined as actual change in the value of indicators being tracked for the respective outcome or output. An outcome or output is considered to have been achieved when its targets have been met.
• Progress towardis defined as actual change in the value of indicators tracked at the next level down in the logic model (i.e. the intermediate outcomes, or their supporting immediate outcomes, or their supporting outputs depending on the level in question), with an explanation of how they are expected to lead to the higher-level outcome.

When there has been no perceptible change in the actual value of indicators at the respective outcome level, go to next level down in the logic model. For example, if there has been no perceptible change in the actual value of indicators at the intermediate outcome level, go to the supporting immediate outcomes and their indicators.

In each case, provide evidence (actual quantitative and qualitative data/information). Explain how these interim accomplishments, at the next level down in the logic model, will, over time, lead to the achievement of the higher level outcome.

Reach

The entity (ies) that a given program or organization is intended to influence, including individuals and organizations, clients, partners, and other stakeholders.^{Footnote 12}

Results

Results are the same as outcomes.

See outcome definition.

Results-based management

Results-Based Management is a life-cycle approach to management that integrates strategy, people, resources, processes, and measurements to improve decision making, transparency, and accountability. Results-Based Management is essential for […] senior management to exercise sound stewardship in compliance with government-wide performance and accountability standards. The approach focuses on achieving outcomes, implementing performance measurement, learning, and adapting, as well as reporting performance. RBM means:

• defining realistic expected results based on appropriate analyses;
• clearly identifying program beneficiaries and designing programs to meet their needs;
• monitoring progress towards results and resources [utilized] with the use of appropriate indicators;
• identifying and managing risks while bearing in mind the expected results and necessary resources;
• increasing knowledge by learning lessons and integrating them into decisions; and
• reporting on the results achieved and resources involved.

The aim of results-based management is to improve management throughout a project and a program life cycle: from initiation (analysis, project planning and design), to implementation (results-based monitoring, adjustments and reporting), and to closure (final evaluations and reports, and integrating lessons learned into future programming). By managing better, you can maximize the achievement of results, that is, the positive changes you set out to achieve or contribute to with your programs or projects.

Results-based monitoring

“… the continuous process of collecting and analyzing information on key indicators and comparing actual results with expected results in order to measure how well a project, program or policy is being implemented. It is a continuous process of measuring progress towards explicit short-, intermediate-, and long-term results by tracking evidence of movement towards the achievement of specific, predetermined targets by the use of indicators. Results-based monitoring can provide feedback on progress (or the lack thereof) to staff and decision makers, who can use the information in various ways to improve performance”.^{Footnote 13}

Results chain

A visual depiction of the logical relationships that illustrate the links between inputs, activities, outputs, and the outcomes of a given policy, program or project.

While some practitioners use the terms “results chain” and “logic model” interchangeably, Global Affairs Canada uses the logic model, along with the outputs and activities matrix and the theory of change narrative, to reflect the complexity of the changes expected from international assistance programming.

See also Logic Model, Outputs-Activities Matrix, and Theory of Change

Stakeholder

Stakeholders include beneficiaries, intermediaries, implementers and donors as well as others such as an individual, group, institution, or government with an interest or concern – economic, societal, or environmental – in a particular measure, proposal, or event.

Target

A target specifies a particular value, or range of values, that you would like to see in relation to one performance indicator by a specific date in the future. Together, the targets established for the various indicators of a specific expected outcome will help you determine the level of achievement of that outcome.

Theory of Change

“Every program [and project] is based on a "theory of change" – a set of assumptions, risks and external factors that describes how and why the program [or project] is intended to work. This theory connects the program's [or project’s] activities with its [expected ultimate outcome]. It is inherent in the program [or project] design and is often based on knowledge and experience of the program [or project design team], research, evaluations, best practices and lessons learned”^{Footnote 14}

At Global Affairs Canada, the theory of change for a specific project is visually displayed in the logic model, which shows the output and outcome levels, and the outputs and activities matrix, which adds activities, and it is fully explained in an accompanying theory of change narrative.

Theory of Change narrative

The theory of change narrative is a crucial complement to the logic model and the outputs and activities matrix. It describes the project’s theory of change and focuses on what is not explicit in the logic model and outputs and activities matrix, such as the logical links between project outcomes and the key assumptions that underpin these links.

It also justifies these links, assumptions and other project-design choices with evidence and lessons learned from other initiatives or practitioners. The narrative should also address any major risks to the achievement of outcomes and describe the measures that have been – or will be – implemented to respond to them.

Triangulation

“The use of three or more theories, sources or types of information, or types of analysis to verify and substantiate an assessment. Note: by combining multiple data sources, methods, analyses or theories, evaluators seek to overcome the bias that comes from single informants, single methods, single observer or single theory studies.”^{Footnote 15}

Ultimate outcome

See Ultimate Outcome above under outcome

Unexpected results / outcomes

A negative or positive change that is not part of the logic model but can be linked to the project. Not to be confused with a risk occurring or with other results not linked to the project.

Work breakdown structure

“the [Project Management Body of Knowledge] describes the work breakdown structure as a ‘deliverable-oriented hierarchical decomposition of the work to be executed by the team.’”^{Footnote 16} The work breakdown structure is a key project implementation tool that can be used to expand on the outputs and activities matrix by breaking the project outputs and sets of activities into corresponding sub-activities or tasks. In other words, the work breakdown structure subdivides the various components of project implementation into lower-level components that provide sufficient detail for planning and management purposes, and tasks that people can actually perform.