How Pathfinder International Uses the Guide to Monitoring and Evaluating KM in Global Health Programs

GHKC's Guide to Monitoring and Evaluating Knowledge Management in Global Health Programs was developed to describe key components of KM activities and help measure outcomes in learning and action. This resource is part of a series of case examples developed by GHKC members highlighting ways the Guide has been used and suggestions for future editions of the Guide.

Contact: Sarah Burns, Knowledge Management Advisor, Pathfinder International (sburns@pathfinder.org)

Pathfinder staff in a knowledge sharing session.

Overview

Pathfinder International has been implementing a variety of knowledge management (KM) activities at the organization level for the past few years. Since 2011, data on KM activities has been captured, analyzed and evaluated to inform changes. Pathfinder adopted process and output indicators from the Guide to monitoring and evaluation knowledge management in global health programs (KM M&E Guide) in addition to indicators for learning and use to evaluate knowledge sharing activities. Pathfinder has used the guide to foster discussion into what should be captured, what reasonably can be captured, what there is time to collect, what there is budget to collect, and what indicators best capture the information that reflect the goals of various KM activities. Beyond the specific use of indicators to track KM activities, in this case example, Pathfinder describes four additional ways that they have used the KM M&E Guide to advance KM.

The goal of Pathfinder’s wide array of KM activities is to provide access to technical information, to promote connections between staff, and to link staff to knowledge to take action, make informed decisions, and improve programs.

Given that the guide was produced by a prominent KM group adds legitimacy to the methods and findings, and licenses evidence-focused staff to be receptive to findings.

Background

Over the past few years, Pathfinder International has been implementing a variety of KM activities at the organization level. Starting with a small library collection and growing to include a technical archive of all past project technical materials, Technical Assistance Libraries (TALs), Knowledge Sharing sessions (i.e. Journal Clubs and Fail Fairs), communities of practice (i.e. Virtual Content Area Teams), an Expertise Finder (i.e. Staff Directory), and learning activities (i.e. After Action Reviews). KM at Pathfinder has evolved from information products and services to learning-focused knowledge exchange and sharing activities. The purpose and goal of the wide array of KM activities are to provide access to technical information, to promote connections between staff, and to link staff to knowledge to take action, make informed decisions, and improve programs.

Evaluating KM Activities

Each year since 2011, data on KM activities is captured, analyzed and evaluated and reported in an annual report. Each year the number of activities and data collected has grown to reflect the expansion of KM activities. Beyond routine data collection, stand alone and time-bound evaluations on specific KM activities, like the Virtual Content Area Teams (Pathfinder’s internal communities of practice), have also been conducted. The goal of collecting the data is to ensure that the knowledge needs of Pathfinder staff are met, whether by improving existing activities or adopting new ones to meet new needs. Two secondary aims are to objectively evaluate what is and isn’t working and to demonstrate the added-value that KM brings to the organization.

Use of the M&E Guide

The M&E Guide has fostered thought into what should be captured, what reasonably can be captured, what there is time to collect, what there is budget to collect, and what indicators best capture the information that reflect the goals of various KM activities. Pathfinder has primarily adopted process and output indicators, although some learning and use oriented indicators have been used in evaluating VCATs and knowledge sharing activities.

The means by which data are captured for the evaluations are many, but include tracking library requests using Excel (including type of request, time it took to complete the request, the requester, etc.), usage statistics of the various online resources (including page views on library resources, searches conducted on archival resources, documents posted to VCATs, etc.), tracking attendance at various knowledge sharing sessions, and bi-annual surveys to all staff that inquire about their use of, and usability of KM resources and activities. The table below shows a small sampling of the indicators used and how they were adapted.

Sample Indicators

Beyond the specific use of the indicators to track KM activities, the KM M&E Guide has also been useful in four main ways:

Reference Tool: The entire guide has been used repeatedly as a reference tool; as a source of indicators for KM staff developing their work plans, and Chapter One in particular, for providing an overview of KM to staff new to the concepts.

Advocacy: The guide has been used as an advocacy tool, since it provides a recognized framework that adds credibility to reports, evaluations, and recommendations. Given that the guide was produced by a prominent KM group adds legitimacy to the methods and findings, and licenses evidence-focused staff to be receptive to findings.

Logic Model: The logic model in the guide has been used to showcase to non-KM staff the value and linkage of KM activities to public health and development objectives.

Capacity Assessment: Although Pathfinder has never formally undertaken a KM assessment, the capacity assessment tool has been useful to KM staff to measure progress year-to-year and to provide a roadmap and guide for where the direction should be headed, particularly because it is customized for global health programs.

Future Suggestions for the Guide

The KM M&E Guide is a valuable resource for KM; however, there are some areas that may be helpful to think about for future versions. First, indicators or guidance around calculating and tracking return on investment for KM activities would be very helpful for getting management buy-in. Second, reference to the differences between program and organizational KM (while acknowledging the title is “in Global Health Programs”) could be beneficial for users of the KM M&E Guide that are more organizationally focused. Third, a more pronounced focus on indicators that can be used to track how KM supports, or is used throughout, program adaptation.  Finally, indicators showcasing how KM activities impact program and health outcomes would help make the case for the use of KM to meet health and development objectives.