How do you evaluate peer-to-peer learning systems?
There are a number of metric categories that are often used to assess and evaluate peer-to-peer learning systems. These metric categories are not listed in any particular order of importance or priority.
-
Networking patterns: Is information flowing efficiently and effectively? Look at the relationship between people and content categories, the network makeup or profile (business unit, job, level, etc.), key brokers and influencers by content category, and the degree of networking across silos. This sort of information will help you to identify communication and decision-making bottlenecks, and groups of individuals that are failing to appropriately collaborate and exchange content.
-
Learning efficiency: How much time are people spending looking for people and information? Look at the time lag between posting content and when content is viewed, the amount of redundant or significantly overlapping content, and the degree to which “informal” content is reused in “formal” content (and perhaps reducing formal content development costs and effort). Information such as this will help you to identify ways to compress learning time, accelerate the uptake of relevant and high quality content, and increase participation in the peer-to-peer learning system.
-
Learning needs: When is the peer-to-peer learning system creating and destroying value? Look at the differences between the learning needs or demand for “formal” and “peer-to-peer” learning (i.e., are some skills best learnt formally?). Slice and dice the findings to better understand the most popular learning needs by job, level, business unit, etc. This information will help learning professionals find ways to save time and money because it will help them to more easily and quickly identify learning needs that require a formal learning solution.
-
Contribution patterns: Are the “right” people contributing at the expected levels, at the “right” times, and using the most appropriate methods? Look at the most active contributors and methods of contribution, busiest days and times for contributing, and the frequency and amount of contributions made by job, level, business unit, etc. Knowing this type of information will help learning professionals target their learning support services and incentive programs. In the end, a peer-to-peer learning system will be better when the “smartest” and “most experienced” people contribute more than anyone else.
-
Content usage patterns: Is the use of learning methods, media, and subject areas at expected levels? Look at the preferred ways to consume various content topics, busiest days and times for viewing content, amount of time spent viewing content and participating in discussion threads and blogs, and preferred way to “find” content. From such information, learning professionals will be able to provide guidance to authors and facilitate the development of more effective learning experiences. Authors will respond more appropriately to employees’ learning styles and preferences, and employees will view and promote more of the learning content.
-
Content quality: Is the “learning community” doing a good job of managing content quality, is there enough “good” content, are there too many unmet learning needs? Look at employee ratings by content category, contributor, and medium, the amount of “inappropriate” or “wrong” content reported by employees, and the amount and type of content with very few hits or views and with a lot of hits or views. By reading this article you hopefully understand how to facilitate the flow of high quality content in the peer-to-peer learning system.
-
Return: Are the benefits of peer-to-peer learning at the expected levels? Look at outcomes such as increased productivity, improved customer service, compressed time to competence, higher reuse of shared information, improved employee engagement, and increased collaboration across silos. British Telecom is realizing $12,000,000 of benefits per annum from its peer-to-peer learning system in terms of cost savings and performance improvements. The return on investment is extremely high because British Telecom used an open source solution that leveraged some existing technologies and software licenses.
-
Opportunity cost: Are we able to do more with less? What costs would we pass up by using peer-to-peer learning instead of formal learning? Look at cost avoidance, travel expenses, reliance on classrooms and trainers, training development budgets, and training course or content maintenance cost. Finding information such as this will help strengthen the case for moving some formal training scope to a peer-to-peer learning system. This sort of action will help reduce the formal training budgets or provide an opportunity to address some of the previously unmet formal learning needs.
Peer-to-peer learning systems are creating more powerful and enduring learning experiences, helping employees establish and leverage social connections to accelerate the distribution and sharing of experiences, content, and guidance, and allowing employees to be more productive, learn faster, and work smarter. Using an appropriate mix of content quality control points will help you to organize and facilitate the uptake of high quality and relevant content. Taking a balanced approach to measurement will appropriately motivate employees to engage in peer-to-peer learning, and produce the data necessary to demonstrate the benefits and continuously improve. What are you waiting for?
