Your Source for Learning
Technology, Strategy, and News
ARTICLES       RSS feed RSS feed


email print

Corporate Executives and the New Secret Weapon: Learning Data

by Carol Leaman

January 20, 2014


by Carol Leaman

January 20, 2014

“Not only can you look to the financial future, you can flag top performers. When you provide them relevant information, specific to their knowledge gaps, when they need it, in a way that’s fast, approachable, and effective, and where they actively realize they’re learning something as they get questions right, it isn’t a stretch to imagine that attraction and retention of the best people is achieved.”

If you ask most C-level executives in 2014 what keeps them awake at night you’ll likely get a combination of the following: sales growth, competitive positioning, and product innovation. However, for the savviest executives today, a rapidly emerging concern is heading to the top of the list: employee motivation and retention.

Data is the key

Today, learning data is the Holy Grail. For years, executives have been searching for a way to truly measure learning outcomes in a cost effective and meaningful way that can enable strategic decisions in the enterprise. Figuring out whether an employee actually “got” what was conveyed to them, and translating that into performance, has been thought of for years as virtually impossible. And looking forward—wouldn’t it be amazing to be able to predict behavior ahead of time based on historical learning data and then tie it to a forecast financial result?

Learning, in 2014 and beyond, is about bottom-line results and quantification. It is also about meeting the deep needs of a mobile and fearless workforce to better equip them with the learning they crave and demand.

Like many other things at work, today’s employee isn’t just sitting back and waiting for professional development opportunities to be offered to them. They expect training and information—on demand, in a way that they can digest it, specific to their knowledge gaps, and using the latest in technology. Who would have thought five years ago that satisfying a demand for continuous education would become a critical element in the attraction and retention of a top-drawer employee?

To date, measurable learning outcomes have tended to be more of the undesirable kind—an employee’s lack of knowledge manifests through negative consequences—they don’t do something they were supposed to, or actively do something they shouldn’t have, and a visible loss results.

Failure to operationalize a procedure that results in a medical accident, for example, is a very visible way to learn that an employee didn’t understand the right way to do something. But many other types of loss that result from lack of learning transfer aren’t so visible—like the employee who fails to follow proper customer-service principles and ends up driving a loyal customer away, or the sales rep who can’t recall the latest product features and pricing in a complex sale and loses to the competition. This results in a negative feedback loop that doesn’t work for the executive who lost the sale, or for the customer, or for the employee who is increasingly looking for positive reinforcement and training.

So, how to invert the learning conversation so that everyone wins?

Creating memory that lasts

The underlying concept is actually really simple—human beings do what they remember. When they don’t remember, they exercise judgment, based on their (sometimes limited or non-existent) experience, the advice of co-workers (who may not remember), or on their best guess. Sometimes they get it right, and sometimes they get it wrong. So what’s the key to real learning? Creating lasting memory!

Imagine being the chief revenue officer of a large multi-national corporation. What if you had the data to show at any given point in time what every single salesperson remembered about product features, pricing, positioning—and you could tie that to their individual sales results? What if you could correlate the success of a salesperson who consistently remembered 90 percent of the information he needed to know against a rep who consistently remembered 35 percent? And what if you had a way to effectively address the second individual’s specific knowledge gaps to close them and improve his performance?

Or, imagine being the VP of operations of a major call center. What cost savings might result from a call center associate who can answer a customer’s questions in half the time of another associate, just because she could instantly recall the appropriate service information?

And as the VP of safety at a manufacturing facility what if you knew that an employee who remembered the correct operational procedures around ladder safety was 80 percent less likely to have an OSHA-reportable incident?

Creating lasting memory is the polar opposite of what yesterday’s training environments achieved. They simply didn’t deliver knowledge in a way that our brains can digest and retain it. The human brain is really good at processing four to five bits of information at a time, and relating them to other concepts and contexts it already knows. Instead, historically, organizations delivered a gargantuan amount of content, all at once, not personalized to the learner and with no reinforcement, and expected employees to find the needles in the haystack and operationalize critical nuggets of information from that point on. They got thrown onto the job, and were expected to know. Except they didn’t, and a lot of visible and invisible mistakes happened.

The most essential thing to focus on is the creation of memory through retrieval practice. Tell, question, and answer. In bite-sized chunks. Every day. That’s it! The act of going back into the brain, and being forced to recall one to five short pieces of information at a time, actually solidifies the neural pathways and creates memory. And as the employee answers, the employer gets to measure knowledge retained at the same time. And what if an employee gets a question wrong? Deliver them the right answer on the spot, and then ask the same question again a few days later. It’s amazing how, when you employ simple retrieval practice, knowledge lift with each iteration of question grows significantly.

On first blush, one can dismiss retrieval practice as creating an environment of rote memorization, which doesn't necessarily address fundamental understanding of concepts. However, memory needs to exist in the first place, so that when the employee actually encounters a situation where they need to apply the knowledge, they have correct recall of the information. It's the first step to making sure they aren't just guessing at what to do. Once they have the opportunity to actually apply the memory, it contributes to understanding and deepens the learning.

How about some examples?

Here’s some data to prove it. In each of the cases below, these Fortune 1,000 organizations use an adaptive learning platform to ask core questions of their employees up to five times in 45 days. They personalize questions to the individual based on their specific job requirements, and adapt to the demonstrated knowledge levels of each person as they answer. In all cases, the information is critical to those employees doing their jobs well, and when an individual demonstrates mastery of information the platform introduces new questions seamlessly and continuously. Each time an employee answers, it measures success and determines how and when to ask the questions again.

Here were the overall results in the first 45 days:

The experience lasts from one to four minutes a day, depending on the number of questions, and is woven into the workday whenever the employee has a few minutes available. Data being gathered and correlated employee-by-employee includes number and frequency of questions answered by individual, by topic, by iteration (1 – 5), by location, by job title, and by success (correct or incorrect).

Financial results

How are the organizations above tying growth in knowledge to a financial outcome for their organizations? Each of the topic areas has a direct correlation to a financial cost.

Let’s take the global retailer above as an example. Like many large retail organizations, this retailer spends hundreds of millions of dollars a year in employee medical accidents and injuries that often result from improperly following procedure. Each type of injury has a typical cost, with back injuries averaging $10,000 – $15,000 per incident. After introducing a daily retrieval-based questioning solution, this retailer has seen a drop in overall incident rates of 20 – 50 percent over historical figures, which they can directly attribute to the type of content and questions they ask every day. Not only are overall incidents now down, and resulting in tens of millions of dollars in savings, but for every incident that occurs the employer has an audit trail to determine how many questions the employee answered in that topic area (if any) and his or her level of knowledge mastery as compared to employees who have no incidents.

Similarly, the complex pharmaceutical manufacturer above was having a very difficult time getting widely disbursed sales professionals to accurately digest and remember specifics around frequently changing products. Simply asking questions and getting them to recall the correct answers ongoing has had a measurable impact on sales results, person by person, which also correlates to their success in answering those questions.

Put simply, employers are discovering that the employees who answer three to five short questions every day, targeted to the core information they need to know for optimal performance, are seeing significant and measurable knowledge growth that translates into improved and measurable performance.

Predictive analytics from learning data

In fact, employers are constantly amazed at how many questions employees initially get wrong in topic areas that are considered to be simple when they start. And they’re relieved when knowledge growth is clearly evident and significant by the fifth iteration. Step by step, this is the way the results grow:

1. Create focused and relevant questions targeted to core knowledge.

2. Question + answer repeatedly = memory (measure how many questions answered, correct and incorrect, by person, by topic)

3. Memory = learning

4. Learning = performance

5. Performance = improved financial results (reductions in incident rates or events, increase in sales—both tied to specific people and how they did in #2. above)

While ongoing measurement and correlation of learning activities to outcomes by employee is now entirely possible, let’s not stop there. This data is a powerful way for executives to also predict future financial outcomes. Let’s take a sales example again—what if you had the data to show that employees who achieve a consistent, minimum success rate of 80 percent answering product and positioning questions achieved 30 percent more sales? For a sales rep with a $1 million quota, that’s an extra $300,000 in revenue. Executives who have that kind of data know how critically important knowledge acquisition is to their overall financial success.

Not only can you look to the financial future, you can flag top performers. When you provide them with relevant information, specific to their knowledge gaps, when they need it, in a way that’s fast, approachable, and effective, and where they actively realize they’re learning something as they get questions right, it isn’t a stretch to imagine that you can achieve attraction and retention of the best people. When employees recognize they are mastering knowledge it makes them feel good, and that good feeling translates into how they feel about the employer (Figure 1).

Figure 1: The outcome curve from regular retrieval practice

The (literal) bottom line

As suggested at the beginning, the most progressive executives are paying attention to learning, and leveraging learning data to provide benefit to both the organization and the employee. Who says you can’t measure learning? It’s a lot less complicated than most people think. Asking a key question, getting an answer, and then reinforcing the correct response repeatedly not only tests memory, but creates it as well. And memory = performance = bottom line results!

Topics Covered

Appreciate this!

Login or subscribe to comment

I'd love to have more specific details about they technology/process used to ask the questions and individualize the response/next question path for each learner.
In 1979 Mina Shaughnessy said, "Ideas are spawned in data." Errors and Expectations, one of the best text books ever on teaching writing! dougb4009
I agree with pluke- I continued reading in hopes of finding some commentary on methods. Anyone have any practical experience they care to share?
I agree, I would love to read more about this. Are there any references or suggestions for further exploration? This has some fantastic possibilities!
Hello everyone, if you want more details about the technology that Carol speaks about in this post. Feel free to get in touch with us at Axonify here:
Related Articles