Indicator Search

Select Logic Model Components and Categories to filter content. To select multiple filters per box, hold the 'control' key while selecting. To export the indicators click the check box to select the item(s) and click the button to export.

Number Returned: 15
Operations
ID Indicator Category Sub Category Logic Model Component Short Definition Long Definition Data Requirements Data Sources Disaggregation Frequency of Data Collection Purposes Issues and Challenges Related Indicators Sample Topics and Questions for Data Collection Instruments Resources Pages in the Guide Published Year Adaptation of Indicator for Specific Purpose (Illustrative Examples) Data Type(s) Intended Use Other Relevant Information Last Updated Date Indicator Snapshots
51 Number of approaches developed, adapted, and/or adopted to facilitate adaptive management of a project, program, or initiative, by type Refers to the number and variety of approaches that have been developed, adapted, and/or adopted to facilitate iterative approaches to learning and adapting while implementing a project, program, or initiative This indicator refers to the number and type of approaches that have been developed, adapted, and/or adopted to facilitate iterative approaches to learning and adapting while implementing a project, program, or initiative to meet performance objectives. This may include the development of new and/or revision, adaptation, or adoption of existing approaches to help foster adaptive management. Quantitative or qualitative data from programmatic records Administrative/programmatic records Annually, after work planning This indicator reflects the planning a project, program, or initiative may take to prepare for and foster adaptive management. Using, adapting, or developing a wide range of approaches can signify a commitment by management to the importance of programmatic flexibility and change that best works for a specific team, project, program, or initiative. Projects, programs, and initiatives must determine the appropriate mix and number of approaches; more approaches is not necessarily better. Although this indicator measures the number of approaches, this should not suggest that it is better to use more approaches. The intent is to measure that there are intentionally selected approaches to facilitate adaptive management. Although a project, program, or initiative may include adaptive management approaches in its work plan, it does not necessarily reflect the use of those sessions in decision making. Indicators in the Reflect subcategory address this common issue and challenge with apply adaptive practices. 2017 Count Wednesday, December 13, 2017
52 Leadership and staff support for adaptive practices Refers to the extent to which leaders and staff demonstrate support for the adaptive management of a project, program, or initiative This indicator refers to the extent to which leaders and staff of a project, program, or initiative support iterative approaches to learning and adapting. This may be self-reported or analyzed as a group or team, and may include awareness of the importance of adaptive practice. Qualitative and quantitative data from responses to questionnaires ​(using Likert scales) ​regarding ​the degree of support from leaders and staff in the use of iterative and adaptive approaches. Periodic surveys Annually, or after specific activities This indicator reflects the perception of staff and leadership in the utility of adaptive practices. Leadership that reinforces adaptive practices is a critical element of adaptive management of a project. Self-reported data may be biased or may not empirically represent the context or practice. On a scale of 1-5, 1 being not a lot, and 5 being a lot, how do you perceive leaders of this project, program, initiative to support the use of adaptive practices for managing the project, program, or initiative? 2017 Categorical scale, qualitative Wednesday, December 13, 2017
53 Number of training sessions or activities focused on adaptive practices that were preplanned in a project, program, or initiative work plan Refers to the number of training sessions or activities focused on iterative approaches to learning and adapting that were preplanned in a project, program, or initiative workplan This indicator refers to the number of training sessions or activities focused on iterative approaches to learning and adapting that were preplanned in a project, program, or initiative work plan. This may include trainings, workshops, and learning events, including knowledge exchange events that are preplanned and budgeted to support the adaptive management of a project, program, or initiative. Quantitative data from programmatic records Administrative/programmatic records Annually, after work planning This indicator reflects the intentional use of adaptive management within a project, program, or initiative. Intentionally identifying adaptive management approaches in a work plan can signify a commitment by management to the importance of programmatic flexibility and change. Although a project, program, or initiative may include adaptive management approaches in its work plan, it does not necessarily reflect the use of those sessions in decision making. 2017 Count Wednesday, December 13, 2017
54 Number of training sessions or activities focused on adaptive practices Refers to the number of training sessions or activities delivered to increase awareness, understanding, or capacity in iterative approaches to learning and adapting This indicator refers to the number of training sessions or activities delivered to increase awareness, understanding, or capacity in iterative approaches to learning and adapting among staff in a project, program, or initiative. Training sessions or activities may be delivered by internal or external experts on topics such as adaptive management; monitoring, evaluation, and learning; complexity-aware programming; facilitation techniques; and collaborating, learning, and adapting. Quantitative data from programmatic records Administrative/programmatic records Annually, after work planning This indicator reflects the need to both train staff in projects, programs, or initiatives in adaptive management and support the implementation of adaptive practices. Without awareness, understanding, capacity, and time, adaptive practices remain ad hoc. Carefully planning and monitoring training and activities can signify a commitment by management to the importance of programmatic flexibility and change. Although a project, program, or initiative may include adaptive management approaches in its work plan, it does not necessarily reflect the use of those sessions in decision making. Although training sessions may be provided, it does not guarantee that the participants were able to internalize the learning and facilitate or use the training materials in the future. 2017 Count Wednesday, December 13, 2017
55 Number of people trained in adaptive practicess Refers to the number of staff trained in iterative approaches to learning and adapting This indicator refers to the number of staff trained in iterative approaches to learning and adapting to improve projects, programs, or initiatives. Quantitative data from administrative records or reports that provide the number of participants, characteristics of participants, gender, and other relevant information Administrative records and reports After specific activities This indicator tracks the initial reach of adaptive practices. It is a simple way to establish a foundation of staff trained in programmatic flexibility and change. Although a project, program, or initiative may include adaptive management approaches in its work plan, it does not necessarily reflect the use of those sessions in decision making. Although training sessions may be provided, it does not guarantee that the participants were able to internalize the learning and facilitate or use the training materials in the future. 2017 Count Wednesday, December 13, 2017
56 Percentage of target staff reporting an improvement in capacity to use adaptive practices Refers to the percentage of target staff reporting an improvement in capacity (knowledge, skills, or abilities) to use adaptive practices This indicator refers to the percentage of target staff reporting an improvement in capacity (knowledge, skills, or abilities) to use adaptive practices for the management of a project, program, or initiative as a results of participating in training or other activities aimed at building capacity in adaptive management. Target staff may include partners. Self-reporting, pre- and post-evaluations, or follow-up surveys should be conducted to determine the extent to which there was an improvement in awareness, understanding, or capacity in iterative approaches to learning and adapting. Quantitative data from pre- and post-tests using survey questions and Likert scales to determine capacity to use adaptive practices and follow-up assessments at three and/or six months to determine knowledge retention; qualitative data can provide greater insight into target user capacity Pre- and post-tests, follow-up surveys Quarterly, semiannually, or after specific activities This indicator can be used to monitor changes in capacity (awareness, knowledge, and skills) in adaptive practices over time (before training/activity and after training/activity). It is a simple way to establish a foundation of staff trained in programmatic flexibility and change. Although a project, program, or initiative may include adaptive management approaches in their work plan, it does not necessarily reflect the use of those sessions in decision making. Self-reported data may be biased and may not empirically represent the context or practice. 2017 Proportion Wednesday, December 13, 2017
57 Number of approaches, methods, tools, or events implemented for reflection and other adaptive practices Refers to the number of adaptive practices (approaches, methods, tools, or events) used to facilitate the adaptive management of a project, program, or initiative This indicator refers to the number of specific adaptive practices, including approaches, methods, tools, or events used to facilitate the adaptive management of a project, program, or initiative. This may include the number of in-person learning events, after-action reviews, lesson-learned workshops, communities of practice, new technologies that facilitate increased ease and frequency of interaction, or other related iterative approaches to learning and adapting. Although this indicator measures the number of approaches, this should not suggest that it is better to use more approaches. The intent is to measure the intentionally selected approaches used to facilitate adaptive management. Quantitative or qualitative data from programmatic records, or self-report of number of adaptive practices conducted, by type; qualitative data to provide greater insight into actual use by staff Administrative records and reports, self-report surveys Quarterly, semiannually This indicator reports the actual implementation of the planned adaptive practices that were identified to be used in the project, program, or initiative. Although a project, program, or initiative has conducted adaptive management approaches, and followed through on their identified work plan activities, it does not necessarily mean that the sessions were of high quality or contributed to programmatic improvements. Projects, programs, and initiatives may find it more useful to measure the proportion of staff using adaptive approaches, however, because the number staff may change overtime (expand and contract) it may be difficult to measure over time. 2017 Count Wednesday, December 13, 2017
58 Number of sessions or activities that include analysis of and/or reflection on monitoring data Refers to the number of sessions or activities focused on reflection and analysis of monitoring data from a project, program, or initiative to inform performance and adjustments This indicator refers to the number of sessions or activities focused on reflection and analysis of monitoring data from a project, program, or initiative to inform performance and adjustments. This may include modifying results reviews, data-quality assessments, or other monitoring and evaluating activities implemented for accountability and to inform decision making to include more learning and reflection. Quantitative data from administrative records or reports that provide the number of sessions or activities implemented with a focus on reflection and analysis of monitoring data Administrative records and reports Quarterly, semiannually, or after specific activities This indicator reports on the implementation of planned adaptive practices identified to be used in the project, program, or initiative that specifically used routine or other monitoring data collected by the project, program, or initiative. Although a project, program, or initiative has conducted adaptive management approaches, and followed through on their identified work plan activities, it does not necessarily mean that the sessions were of high quality or contributed to programmatic improvements. Quality can be assessed through user satisfaction surveys (see indicators 24 to 28); actions taken to make programmatic improvements can be assessed in an internal assessment. 2017 Count Wednesday, December 13, 2017
59 Number of actionable recommendations identified or collected to inform project, program, or initiative performance Refers to the number of actionable recommendations to inform project, program, or initiative performance or adjustments that were collected from the use of adaptive practices This indicator refers to the number of actionable recommendations to inform project, program, or initiative performance or adjustments that were identified or collected from the use of adaptive practices or from sessions or activities focused on reflection and analysis of monitoring data. The number can be used to calculate a percentage of action taken. Quantitative and qualitative data from the review and analysis of meeting minutes, reports, and other documentation to determine how many recommendations are actionable. Administrative records and reports Quarterly, semiannually, or after specific activities This indicator reports the number of recommendations collected that are actionable. Recommendations should identify a point of contact or timeframe for its use, rather than be a general statement that provides no next steps. This indicator should help staff to reflect on whether the stated recommendations can be used, by whom, and by when. By collecting the number of actionable recommendations, a percentage of actions taken can be calculated during an internal assessment (see Act subcategory). Although a project, program, or initiative has conducted adaptive management approaches, and followed through on their identified work plan activities, it does not necessarily mean that the sessions were of high quality or contributed to programmatic improvements. Review and analysis of documentation can help determine how many recommendations are actionable, such as actions taken within the scope of the project, program, or initiative; responsibility and next steps are clearly documented; and budget and time allocated, and so on. Teams can determine how best to define "actionable" to meet the project, program, and/or initiative needs. 2017 Count Wednesday, December 13, 2017
60 Percentage of intended users who are satisfied with trainings, approaches, or events focused on adaptive practices Refers to the percentage of intended users who are satisfied with trainings, approaches, or events that focus on or promote adaptive practices This indicator measures the percentage of intended users who are satisfied with trainings, approaches, or events that focus on or promote the management of a project, program, or initiative through adaptive practices. A satisfied user indicates that intended needs and expectations were met. User feedback should inform future activities. Multiple data points will monitor change in user satisfaction over time. Quantitative data from self-reported surveys or questionnaires using Likert scales to determine user satisfaction with trainings, approaches, or events focused on adaptive practices; qualitative data can provide greater insight into user experience, attitudes, and preferences Surveys Semiannually or after specific activities The aim of this indicator is to gauge user satisfaction with the trainings, approaches, and events that were selected for adaptive practices purposes in the project, program, or initiative. This indicator does not measure the quality of the trainings, approaches, or events, as it looks at self-reported satisfaction. It is possible for staff to be highly satisfied with a training or event, but for that training or event not have impact on the improvement of the project, program, initiative, or decision-making processes. 2017 Categorical scale, proportion, qualitative Wednesday, December 13, 2017
61 User rating of usefulness of content/outputs produced from the use of adaptive practices Measures the perceived quality and/or relevance of content/outputs produced from the use of adaptive practices to inform the management of a project, program, and/or initiative User rating of quality and/or relevance of content/outputs produced from the use of adaptive practices to inform the management of a project, program, and/or initiative. “Content” refers to the outputs, such as action steps, that emerge from a training, workshop, session, or event. "Relevance” refers to whether intended users find the information or knowledge applicable and important to their work. Multiple data points are used to monitor change in user reporting of the usefulness of adaptive practices over time. Quantitative data from self-reported surveys or questionnaire using Likert scales to determine the quality or relevance of participating in activities intended to training in or use adaptive practices for program implementation Surveys, after action reviews Semiannually or after specific activities This indicator measures the user rating of the quality of the outputs from adaptive practices used in the project, program, or initiative. It is a way to measure the usefulness or quality of the practices. This indicator does not measure the quality of the trainings, approaches, or events, as it looks at self-reported usefulness. It is possible for staff to believe that the practices/outputs were of high quality or relevant, but for those practice/outputs not to have impact on the improvement of the project, program, initiative, or decision-making process. Qualitative data can provide greater insight into user experience, attitudes, and preferences and how quality can be improved. 2017 Categorical scale, qualitative Wednesday, December 13, 2017
62 Number of instances when target users report that their projects reused or adapted previously captured knowledge/resources to design or start a project, program, and/or initiative Refers to the use of previously captured knowledge for decision making while designing or starting a project, program, or initiative This indicator refers to the use of previously captured tacit and explicit knowledge for decision making while designing or starting an activity, such as a project, program, or initiative. This may include accessing previous after-action reviews, reviewing previous project documentation, conducting a peer assist with actors previously engaged in similar work, or conducting before-action reviews with the team. It can apply to work-related decisions and practices at project, program, initiative, and individual levels. Anecdotal evidence may be included regarding whether a consultation has occurred or resulted in actionable recommendations. Quantitative and qualitative data from key informant interviews or focus group discussions, used to determine when previously captured knowledge or resources were use, reused, or adapted for a specific activity Key informant interviews, focus group discussions, after-action reviews Semiannually or after specific activities This indicator measures actions taken to put reflection and knowledge into practice. By asking open-ended questions, the indicator captures not just the number of instances of the material being used, but also the way it was used, how it was adapted, and how it was useful in the design or start-up of a project, program, or initiative. This indicator addresses a common challenge in adaptive management—taking the time to use previously captured knowledge and resources. This number can be used to calculate a percentage. Finding time for discussion and analysis can be challenging for a project, program, or initiative. This indicator is mostly used to evaluate the adaptive practices selected, not the program itself, which can result in a low prioritization of this indicator. Qualitative data collection can provide additional insights on the meaning of this indicator, and is strongly recommended because of potentially divergent points of view of what constitutes use, reuse, adaptation, and/or action. 2017 Binary (y/n), count, qualitative Wednesday, December 13, 2017
63 ​​Number of instances when target users report the application of knowledge captured through reflection to inform decisions or to take corrective action Refers to the use of knowledge in decision making while taking corrective action, or attempting to improve a project, program, or initiative This indicator refers to the use of tacit and explicit knowledge in decision making while taking corrective action or attempting to improve an activity, such as a project, program, or initiative. It measures the use of actions or outputs from reflective learning sessions in decision making or practice, which may include work planning, requests for technical assistance, modifications in project implementation or management activities, data review meetings, or identification of additional project or program needs. It can apply to work-related decisions and practice at the project, program, initiative, or individual level. Quantitative and qualitative data from key informant interviews or focus group discussions to determine when decisions were made based on knowledge, information, or insights captured during reflection Key informant interviews, focus group discussions, after action reviews Semiannually or after specific activities The aim of this indicator is to measure actions taken to put reflection and knowledge into practice within the project, program, or initiative. By asking open-ended questions, the indicator captures not just the number of instances that knowledge was used to inform decisions or take corrective action, but also the way it was used, how it was adapted, and how it was useful. Finding time for discussion and analysis can be challenging for a project, program, or initiative. This indicator is mostly used to evaluate the adaptive practices selected, not the program itself, which can result in a low prioritization of this indicator. Measuring this indicator should be carefully considered as it could require considerable level of effort. Whether projects will want/need to measure this is debatable, but it still is an important measure of the preparation and reflection that resulted from previous subcategories. Users may want to limit the actions taken from one specific adaptive practice, such as an after-action review, in order to make this indicator manageable. Qualitative data is strongly recommended. 2017 Count, proportion Wednesday, December 13, 2017
64 Degree of change in project norms or behaviors conducive to evidence-based decision making and action, as reported by target users Refers to the extent to which target users report a change in project or program norms and behaviors toward those conducive to evidence-based decision making and action This indicator refers to the extent to which target users report a change in project or program norms and behaviors toward those conducive to reflection and evidence-based action as an outcome of the use of adaptive practices. This may include changes in beliefs, opinions, and perceptions of both the project, program, or initiative and the individuals themselves regarding the value and benefit of the adaptive management of project, programs, or initiatives. Quantitative data from surveys or questionnaires using Likert scales to determine degree of change in norms or behaviors, self and others; qualitative data from key information interviews and focus group discussions on specific norms and behaviors reported to have changed and why, and the implications of those changes Surveys, key informant interviews, focus group discussions, after-action reviews Annually The aim of this indicator is to measure the usefulness of the adaptive practices in changing the culture of the project, program, or initiative to one that values evidence-based decision making and action. It is difficult to measure norms; there are a lot of quality issues and little consistency in how norms are measured. Finding time for discussion and analysis can be challenging for a project, program, or initiative. This indicator is mostly used to evaluate the adaptive practices selected, not the program itself, which can result in a low prioritization of this indicator. It also is challenging, but not impossible, to get a good baseline of the project norms prior to the use of adaptive practices. Evidence-base for Collaborating, Learning, and Adpting (eb4cla): https://usaidlearninglab.org/eb4cla 2017 Categorical scale, proportion, qualitative Wednesday, December 13, 2017
65 Degree that adaptive practices have contributed to the objectives of a project, program, or initiative Refers to the extent to which taking an adaptive approach has contributed to achievement of project, program, or initiative objectives This indicator refers to the extent to which taking an adaptive approach has contributed to achievement of project, program, or initiative objectives. Specifically, that an adaptive practice has made an impact on a project, program, or initiative. Generally, projects, programs, or initiatives should not be required to show the unique contributions of adaptive practice on their objectives; instead, they should indicate the use of tools, approaches, and processes demonstrated to contribute to the more efficient and effective delivery of objectives. Quantitative data from surveys or questionnaires using Likert scales to determine degree of change in norms or behaviors, self and others; qualitative data from key informant interviews and focus group discussions on specific norms and behaviors reported to have changed and why, and the implications of those changes Surveys, key informant interviews, focus group discussions, after-action reviews, research studies Once, at the conclusion of the project, program, or initiative The aim of this indicator is to measure the extent to which adaptive practices improved and/or influenced project, program, or initiative outcomes. Allocating project funds for this type of study, or building it into the evaluation of the project, program, or initiative, can be challenging due to competing priorities. It is likely that there would not be a comparable project, program, or initiative that could be evaluated as a control against the project, program, or initiative that did use adaptive practices, so the study of the difference would be minimal or extrapolated. However, one could compare similar projects, one that used adaptive practices, and one that did not, and compare outcomes. Ideally, the extent of this change would be shown through the collection of stories, cases, or examples about how adaptive practices contributed to a project, program, or initiative. It also is challenging, but not impossible, to get a good baseline of the project norms prior to the use of adaptive practices. Evidence-base for Collaborating, Learning, and Adpting (eb4cla): https://usaidlearninglab.org/eb4cla 2017 Categorical scale, qualitative Wednesday, December 13, 2017