If we want to improve our corporate training, we need to constantly assess our programs in order to determine what is working well and what is not. And so I was a bit distressed when, during my session at the recent Learning Solutions Conference, several audience members said, “I am too busy training. I do not have time to do research.”
This remark got me thinking and I want to share some ideas to help you reconsider the importance of building assessment into your organization’s training.
Why do we assess?
Let’s start with an analogy. My wife Jane complains about my driving. It’s not that I am careless or aggressive. The problem is that I am usually in a hurry and I don’t take the time to check a map or GPS before I leave the house. Like many men, I trust my intuition and assume I know the right way. Unfortunately, my best guess is often wrong and we end up hopelessly lost.
Training is like that. We are so busy training, so busy creating one lesson after the next, that we do not take the time to assess where we are and whether we are making progress. And when our employees fail to thrive, we just train them some more.
Doesn’t it make more sense to slow down, take stock of our position, and make strategic decisions that will help us find the best route to our destination?
Intuition is not enough
There is a long history of people assuming that they were doing good work when they were not. For example, a few years ago, a group of well-intended folks delivered seminars warning college women of the risks of eating disorders. The trainers delivered workshops on campus after campus, always too busy with their good work to assess their effectiveness. When they finally did do systematic experiments they were shocked to discover that on the campuses where they made presentations there was actually an increase in the prevalence of these disorders. Tragically, it turned out that their workshops destigmatized eating disorders and made women more likely to partake.
Is it possible that your training is having no effect on learners, or worse, that it is actually detrimental to the good of the company? The answer is “Yes” it is possible, and until you conduct systematic assessment you will have no idea whether you are really helping the organization reach its goals.
When should we assess?
If you decide that you want to deliver assessments, the next question is “when” these assessments ought to take place. I polled my audience in Orlando about when, if ever, they delivered their assessments. Here are the results:
|
Never: |
65% |
|
Immediately after training: |
30% |
|
In the days and weeks after training: |
5% |
As you can see, only about one-third of the audience actually conduct assessments, and of those that do, almost all of them conduct the assessment immediately after the training takes place. One trainer explained, “We have the learner available right after training so it makes sense to do the assessment at the same time.”
But does it make sense? Is an immediate assessment going to give you the meaningful data about what people are likely to retain? To best answer this question, I need to give you a bit of background on how memory works.
The three stages of memory
Most neuroscientists agree that memory consists of three distinct stages: sensory memory, short-term memory, and long-term memory.
Sensory memory
Your sensory memory is a buffer that stores all of the sensory information you receive. Sensory memory seems to store everything you perceive (sight, sound, touch, for example), but this storage lasts at most only a few seconds. The fact is that most of this sensory information is unimportant to you, and if you pay no attention to it, it quickly fades away, never to be remembered. On the other hand, if you do focus on a particular aspect of your sensory memory, the full range of the information is available to you.
To illustrate, assume you are watching your favorite TV show while at the same time, your spouse is going on and on talking to you about his or her busy day. Suddenly, you hear them say loudly, “Are you even listening to me?”
You snap away from the TV and say, “Of course I am, dear.” And in a surge of cognitive effort you search your sensory memory and you recall the last sentence or two. “Uh... You were just saying that ‘George is not finishing his homework and you are getting frustrated with him.’”
The fact is that you were NOT really paying attention to what your spouse was saying. But the information was stored within your sensory memory and by shifting your attention to it, you were able to retrieve it before it faded. Whew!
Sensory memory is the shortest-term element of memory and it decays very quickly. For example, iconic memory (the memory of vision) usually decays within a half-second, whereas echoic memory (the memory of sounds) can last up to three or four seconds.
As you can see in Figure 1, if you ignore information in your sensory memory, it will fade. If you “pay attention” to it, it is transferred into your short term memory.
Figure 1: Memory components
Short-term memory
Short-term memory (STM), also known as primary or active memory, is the information we are currently thinking about. Our short-term memory has the ability to contain small amounts of information in a readily available state for short periods of time. In a famous article titled The Magical Number 7, plus or minus two, George Miller argued that people can usually hold only about seven items, for some it is a bit more, for others a bit less. (Recent research has adjusted the number down a bit).
We can usually maintain information within STM for as long as we continue to actively process or rehearse it. However once we stop actively processing the information, perhaps because we are distracted by another thought, the original memory trace can fade in a few seconds.
To illustrate STM, assume you are being introduced to a series of new junior executives in your organization. Miller’s theory predicts you will be able to actively store and rehearse about seven names at a time. When you are introduced to the eighth person, however, you then exceed the capacity of STM and something has to be pushed out of STM. If the ejected item has been insufficiently rehearsed when it is pushed out of STM that item will be lost. On the other hand, if that item has been sufficiently rehearsed, the memory will be encoded and transferred to long term memory.
Long-term memory
Long-term memory (LTM) is where encoded information is stored permanently. Unlike sensory and short-term memories, long-term memory can store (we think) unlimited amounts of information and can do so permanently. As trainers, our goal is to get our ideas transferred into the learner’s LTM and to ensure that the learner is able to retrieve that information when they need it.
So when should we assess learning?
The value of assessments depends on when you deliver it. For example, if your goal is to determine whether the learner understood complex material and you want to know if they understood the distinction between “consensual leadership” and “directive leadership,” it is entirely reasonable to ask them a quiz question immediately after training that requires them to demonstrate the distinction.
On the other hand, however, if you want to determine whether a learner is likely to remember and utilize information, an immediate assessment is near to useless. The problem is that an immediate assessment simply measures whether the information is contained within the learner’s STM and this has little correlation with whether it will be transferred to LTM. As a result, an immediate assessment leads to the false impression that people have learned a lot. If you want to predict long term retention, you need to delay the assessment, for an hour or even a day, so that you can know that you are really testing a person’s long term memory.
Next month, we will look at other ways that one can use assessment and how best to deliver it.
Digging deeper
If you would like to have your memory of this article reinforced, send an email to ELGApril2015@aklearning.com. You will automatically receive a series of boosters on this article. The boosters take only seconds to complete, and they will profoundly increase your ability to recall the content of these articles.
If you want to learn more about the mishap with the eating disorder training, here are some good references:
- T. Mann et al. (1997). Health Psychology, 16, 215-225
- E. Stice & H. Shaw (2004). Psychological Bulletin, 130, 206-227.
- C.B. Taylor et al. (2006). Archives of General Psychiatry, 63, 881-888.
A short cut for delivering assessment
If you really need to deliver training and assessment during a single session, the trick is to find a way to fully occupy the learner’s mind so that you can be sure that the STM is purged. Memory researchers use the Brown and Peterson procedure wherein students count backward from 1,000 by threes for about a minute. Within a training environment, however, a more realistic strategy might be to give the learner a second training experience before assessing them on the first.
Is the three-stage model of memory true?
The three-stage model is compelling and easy to understand but technically, it is not “true.” No one knows how the brain works and the three-stage theory is just a representation of our best guess. Our knowledge of the brain will continue to grow, and you can be sure that this model will be replaced by one that does an even better job explaining the data. All models are wrong—but some models are useful for a while.