Your Source for Learning
Technology, Strategy, and News
ARTICLES       RSS feed RSS feed

Case Study: How a 3-Year Project Led Us To Scenario-Based Course Design

by Mike Dickinson

July 26, 2010

Feature

by Mike Dickinson

July 26, 2010

“Writers should make scenarios compelling by using suspense and, if at all possible, real-life dilemmas that aren't trivial. In fiction, that is done through exaggeration and conflict. With real-world training you can't necessarily do that, but you can uncover the tough dilemmas that give people a great deal of trouble.”

We are now in the third year of an intensive annual compliance-training program for employees of our company, which manufactures and distributes electric wheelchairs, mobility scooters, wheelchairs and motorized chairs. Most of the training has been online, and sometimes in the form of a game show using third-party software to make instructor-led delivery much more fun and engaging. An experienced Medicare compliance consultant labeled our online training the best he's ever seen. I think he was very generous, but, hey, we'll take the compliment!

We have faced several challenges, including quickly evaluating and mastering authoring systems and tools, meeting prescribed learner-contact times (three hours per year), and tracking and reporting completion, just to name a few.

In our most recent round we used a more scenario-based approach, and as a result we have learned some lessons about scenario design, independent of the actual authoring system. Eventually we were able to devise a template to elicit scenario-worthy content from SMEs.

We are a company of about 2,500 employees with annual revenue expected to be $450 million in 2010. Geographically, our home office is located in south-central Texas where we have our sales call center and corporate offices – about two-thirds of our employees. The rest of our employees are spread around the country, operating out of their homes or local distribution centers or retail stores.

Many of our customers are of Medicare age (64+), so we are considered a Medicare provider. Thus, Medicare compliance has always been an integral part of our policies and training, including a Compliance Department that conducts continual internal audits and review boards. In 2007 our compliance program was enhanced significantly, as was the associated training.

Year 1: Approach

The initial requirement was to design, develop, and deliver compliance training to all employees within 120 days due to the newly enhanced compliance program. Employees were required to take two to four hours of training depending on their job. We chose online training as the prudent way to reach all employees nationwide in a relatively short time.

Tools

Before this requirement arose, I had been disenchanted with the authoring system we were using at the time. It was robust and had lots of flexibility, but it was difficult to learn and took a lot of effort to build engaging interactions. So I had been keeping a continuous watch for an authoring tool that would do more of the heavy lifting, enabling us to focus on design while it provided more out-of-the box interactions. Enter Articulate Presenter.

We had just purchased the Articulate Presenter suite when this concerted compliance-training project began, and it was just the tool I needed! (I was an e-Learning shop of one at the time.) Articulate was based on PowerPoint so I could build screens quickly. It had just enough built-in interactions with its learning games and synchronized animations to add visual and aural interest. In addition, Articulate made narration very easy to do. This was important because we elected to use narration and forced navigation as the way to ensure each learner spent the minimum required time in the training. I don’t like doing that, but we saw no other viable solution.

Content development and delivery

We broke “compliance” up into sub-topics that included federal laws and guidelines, details about our enhanced compliance program including the code of conduct and internal policies and procedures, and other Medicare compliance resources.

Articulate Presenter comes with a few learning games that we used at frequent intervals to add some fun while reinforcing key points. A favorite was the word game that is similar to hangman. (See Figure 1.) The learner clicks letters to fill in the missing word, and the author can set a time limit for each question.

 

Clue: Aswer, question and answer reveal Tile game

Fig. 1: Sample Articulate word game slide

 

The learner gets audio applause if they answer correctly and a “raspberry” sound if they miss a question. I was stopped in the hallways many times by employees telling me how much they enjoyed that applause! For new hires we sometimes ran the course in a group setting, rotating drivers of the mouse among classroom participants. I can tell you that the applause along with the desire to not get the raspberry sound or let the time run out were strong motivators to answer each question quickly and correctly!

Course structure

One of the decisions we dealt with at the beginning of this project was how to package the online training. Should we have a couple of two-hour courses, or break it up into smaller pieces? At the time, our LMS wasn’t bookmarking reliably enough to have confidence in it, so we created eight 30-minute courses. This gave learners a sense of forward movement, but it also created some confusion, and it made tracking and reporting a huge chore.

How did it go?

The heightened awareness of compliance was noticeable, and course critiques (Kirkpatrick Level 1) were very favorable. Thank goodness, because I felt those 120 days of development drew on all the skill and cunning I had developed as an instructional designer over the years!

One interesting item about training evaluation: What would be a good Kirkpatrick Level 3 or Kirkpatrick Level 4 measure for this compliance training? Would you expect fewer compliance incidents, or more? In our case, the number of calls to the internal compliance hotline increased noticeably in the weeks right after the training. We viewed that as a positive response. The training stressed the importance for the company of being a self-correcting organization, it clearly described the process for using the hotline, and it reiterated the policies of anonymity and non-retaliation for identifying potential issues to the hotline. Employees took that to heart and became much more proactive than they had been before, to include self-reporting if they realized after the fact they may have committed a compliance error.

Year 2

We ran some refresher training near the end of Year 1. At our corporate headquarters, we conducted several standup sessions in a large classroom, delivered in person by our Corporate Compliance Officer (CCO). While employees valued the chance to hear his insights and ask questions firsthand, we also saw some drawbacks inherent in the live presentation format: learners could be passive and the time commitment became a burden for our CCO.

Content development and delivery

For Year 2, we wanted to try something different for certain groups. We ran online training for those in the field and for certain groups at the corporate office, much like our initial training, but with updated content. However, our largest group of employees is also a group with daily exposure to compliance risk: our inside sales and support staff. They spend all day on the computer and phone, and they tend to be extroverted and competitive by nature. We wanted to appeal to these traits while reducing the burden of a two-hour chunk of training, and at the same time we wanted to help them internalize the training.

Tools

After searching the internet again, I found a wonderful program called Game Show Presenter by Tom Bodine (no connection to Articulate Presenter). Game Show Presenter, which you can see in Figure 2, can be purchased to run in a group setting and/or online. In order to run the game show we needed not only the program, but also a set of recommended buzzers, a laptop computer, and speakers.

 

grid with learning topics and points

Figure 2: Sample game show round

 

We had a fortunate mix of circumstances: Our sales teams have ten people each, and we had a conference room very close to the Sales work area that would hold four teams. We came up with a “passport” scheme called Destination: Compliance. It consisted of three parts: a one-hour game show competition, a facilitated discussion within each sales team, and an online quiz. We scheduled four teams at a time for the live game show and gave the winning team a commemorative tee shirt. You wouldn’t believe the spirit this generated! You could hear the game show from way down the hall.

Course structure

By its nature, the game show consisted of standalone questions. With help from our Compliance Department, we developed these questions carefully to include the feedback and any elaboration. (You can display a debrief screen after each question if you like.) In addition to factual questions, many were mini-scenarios, eliciting what an employee should do in a given situation.

Each team rotated their players through the buzzer hot seat for full and equal audience participation. Teams were encouraged to coach their hot-seat person and that sometimes got very lively with hardly anyone staying in their seat! We used the online version of Game Show Presenter for make-ups.

The game show was a big hit, and again, a piece of software helped make us look like heroes. (We were now an e-Learning shop of two, plus one intern who subsequently became an employee.) Other departments have used Game Show Presenter as a fun way to do refresher training, and we’ve started using it in one of our entry-level courses. You can always tell when it’s in use by the noise filtering out of the room and down the hall.

How did it go?

We went into the Year 2 round of refresher training wanting to reinforce numerous details about compliance and our enhanced compliance program. So we used the same questions for the game show and the online quiz (with answers randomized – remember that this is a “Jeopardy”-type game).

As teams arrived for the game show we gave each person a note-taker that had all the questions and responses, without the correct answers marked. Looking at the group from the front of the room during the game, everyone was not only engaged but intensely so, note-taker and pen in hand. They made sure they knew the correct answer to each question! Scores on the online test were very high, which was our goal. We weren’t out to stump people; rather, we wanted to be sure they knew the material well.

After employees finished all three parts of their Destination: Compliance journey we asked what they thought. Figure 3 shows their responses.

 

horizontal bar chart

Figure 3: Participant responses to Level 1 evaluation of Destination: Compliance

 

We tried one other thing in Year 2. For a company-wide piece of the refresher training that was online, we wrapped a scenario around the online training, creating some suspense early in the training, alluding to it throughout, and not resolving it until the end. This became a precursor to the approach we took in Year 3, based on the feedback shown in Figure 4.

 

horizontal bar chart

Figure 4: Evaluation of blended approach


Topics Covered

(65)
Appreciate this!
Google Plusone Twitter LinkedIn Facebook Email Print
Comments

Login or subscribe to comment

Thank you so much for sharing this!
You're very welcome!
Just yesterday I read an article, Creating e-Learning That Makes A Difference by Ethan Edwards of Allen Interactions. This case study provided examples of what the "Makes a Difference" article recommended! Thank you for sharing what you did, how it worked and participant responses. Very encouraging!
This is a great article! It both affirmed what I've seen when developing scenario-based online training, and provided me with some things to think about for future courses. Thanks for sharing!
Excellent summary of process & impact. We used a decision-tree set of scenarios as an add-on to the ethics training provided for Dell employees. At first, there was concern about the time spent working through each scenario, but in practice, learners stayed more engaged and felt they spent less time on the scenarios than on the other sections -- altho the reverse was actually true! People will spend the time willingly if they are actively engaged in applying the lessons learned.
Excellent article. And timely too.
Really appreciate the effort that you took to put together this article. You have helped me learn in 15 minutes what you learned in 3 years! Thank you!
Related Articles