Indicators That Measure Adaptive Practice

Overview

Donors, clients, and implementing agencies have recognized that programmatic change does not usually follow a predictable linear path, and seek ways to implement more systemic solutions to difficult, interrelated, and complex problems. The global health and development sector has recently seen a rise in iterative and adaptive approaches to program design, implementation, and evaluation. These adaptive approaches encompass a wide variety of approaches, such as systems thinking, human-centered design, adaptive management, and lean and agile management, all of which share a focus on “multiple-loop” learning and experimentation. For our purposes, adaptive practice refers to the practice—preparation, implementation, and monitoring—of adaptive management, which is defined by USAID as “an intentional approach to making decisions and adjustments in response to new information and changes in context” (USAID, 2017).

Recent literature suggests that adaptive practices should be included in project designs because they allow a project to improve throughout implementation, respond to unforeseen challenges, redirect resources to unexpected needs, and build on collective knowledge to produce the best possible development outcomes (Dexis Consulting Group, 2017). In order to address the dynamic contexts in which they work, donors and project managers want to use the increasingly available data and information to make better-informed choices. They also need to ensure that the knowledge generated by project staff is used to inform decisions that can increase the project’s benefits to those it seeks to serve; KM helps them do just that.

The increased attention on adaptive practice and applied learning has had a direct impact on the field of KM. KM supports adaptive practice the way it supports programs and organizations in general, by ensuring everyone has access to the right information, at the right time and in the right format, to help them make informed decisions and do their jobs. It also provides input into the methods and activities used to capture, share, and use knowledge to iterate and adapt programs. In fact, to many KM practitioners, adaptive practice is not a new concept, but a natural extension of using knowledge effectively. At its core, effective KM aims to develop, strengthen, and institutionalize practices that improve team and project performance and efficiency to meet development objectives, and adaptive practice is a natural outcome of these activities.

Section objective:

The objective of this section is to integrate adaptive practice across the KM logic model, primarily by modifying existing indicators and adding new indicators that monitor the preparation, implementation, and support taking action based on new information and knowledge. As with most of the indicators in this guide, the proposed indicators for adaptive practice are the means through which projects and programs achieve their goals more effectively.

Three principles inform this section on adaptive practice:

  • Adaptive practice should be systematic, planned, and intentional, but should not burden implementers with additional work.
  • Adaptive practice must be integrated into project design and implementation, including KM and monitoring and evaluation.
  • The measurement and success of adaptive practice will ultimately be determined by decisions made or actions taken based on new information.

Using indicators in this category:

These indicators are intended to help KM staff incorporate adaptive practice into their support of global health programs and projects. The indicators listed are primarily quantitative and are not comprehensive. In creating this section, the authors reviewed all new and existing indicators in the guide for overlap with the adaptive practice indicators. Existing indicators were edited to include adaptive practice principles; where gaps existed, new indicators were created that specifically highlighted adaptive practices.

The indicators in this guide are meant to help development practitioners measure the use of adaptive practice in a project or organization, not necessarily the contributions of those practices to project objectives. The evidence base for adaptive practice is growing and funding mechanisms have been created to support the study of the impact of adaptive approaches on development.

We recognize that adding more indicators to a program, project, or initiative can be not only difficult, but often impossible or undesirable. These indicators should act as guidance for project staff tasked with supporting or reporting on adaptive or course-corrective actions taken by a project. We acknowledge that these indicators may make adaptive practice yet another activity to be completed. Despite this, these indicators can and should be used to complement regular reflection on a program’s progress toward meeting its objectives, and what types of management practices and systems most effectively help or hinder a program’s success.

We recommend, therefore, that users of this guide select only one or two indicators per subcategory that are best suited to their project, program, or organization, and measure them as part of their regular monitoring and evaluation activities. For example, indicators could be selected based on the goal of further embedding adaptive practices into a program. In this case, selecting an indicator from the Prepare section will ensure there is at least one intentional adaptive practice method selected for the program.

Adaptive practice often requires donors, managers, implementers, and evaluators to shift their thinking about programmatic change and adaptation. This entails making the task of adaptive practice less intimidating by focusing on what can be changed right now, creating an environment where adaptation and course correction is incremental and frequent—rather than radical and reactive—and where adaptive practice can take many forms, such as intentional and systematic or organic and spontaneous. Not every team will be ready to embrace adaptive practices immediately. Project staff using these indicators will need to build on the readiness of certain teams and leaders allowing for diffusion of adaptive practice through attraction rather than as a requirement.

Subcategories

Indicators that measure adaptive practice are grouped into three subcategories: 1) prepare, 2) reflect, and 3) act. Altogether, 15 indicators are mapped to these subcategories (indicators 51 to 65).     

Subcategory (Area)  Description
Prepare    The selection, development, and adaptation of approaches for adaptive practice, and the requisite capacity and resources to use those approaches within a project, program, or organization. 
Reflect    The implementation, delivery, and evaluation of selected, developed, or adapted approaches for adaptive practice within a project, program, or organization. 
Act    The application and translation of new data, information, or knowledge acquired by using the adaptive approaches into actions that can improve a project, program, or organization. 

Indicators

ID Indicator
51 Number of approaches developed, adapted, and/or adopted to facilitate adaptive management of a project, program, or initiative, by type
52 Leadership and staff support for adaptive practices
53 Number of training sessions or activities focused on adaptive practices that were preplanned in a project, program, or initiative work plan
54 Number of training sessions or activities focused on adaptive practices
55 Number of people trained in adaptive practicess
56 Percentage of target staff reporting an improvement in capacity to use adaptive practices
57 Number of approaches, methods, tools, or events implemented for reflection and other adaptive practices
58 Number of sessions or activities that include analysis of and/or reflection on monitoring data
59 Number of actionable recommendations identified or collected to inform project, program, or initiative performance
60 Percentage of intended users who are satisfied with trainings, approaches, or events focused on adaptive practices
61 User rating of usefulness of content/outputs produced from the use of adaptive practices
62 Number of instances when target users report that their projects reused or adapted previously captured knowledge/resources to design or start a project, program, and/or initiative
63 ​​Number of instances when target users report the application of knowledge captured through reflection to inform decisions or to take corrective action
64 Degree of change in project norms or behaviors conducive to evidence-based decision making and action, as reported by target users
65 Degree that adaptive practices have contributed to the objectives of a project, program, or initiative