Albert Einstein, in his book of essays Out of My Later Years, made the observation that, “Perfection of means and confusion of goals seem — in my opinion — to characterize our age.” This is just as applicable to e-Learning today as it was to politics and physics in Einstein’s time.
One way to tell how much experience someone has with e-Learning is to notice the questions they ask. Is their concern with means or with goals? New practitioners tend to ask, “Which authoring tool or LMS or LCMS should I use?” But those who have survived their first project or two have questions about the role of subject matter experts, how to deal with “issues” over who makes key decisions, or solving problems with the constraints: how can they get enough budget, time, and staff?
Managers, designers and developers with even more experience want to know how to connect their e-Learning to the important objectives of the business. Eventually they only ask about business. They have figured out that the key to success is to discover what keeps the Chief Executive Officer or their customers awake at night — and to offer e-Learning applications that align with those concerns.
The basic mission of e-Learning development, if I might offer a definition, is to provide a system of instruction and guidance to achieve a valued result through human accomplishment. The key words here are “...to achieve a valued result....” The development team has to understand this long before thinking about storyboards, ActionScript, or SCORM. That understanding makes the difference between zeros and heroes.
Last week, I gave a quick overview of the challenges in evaluation, or quality control, of e-Learning. Evaluation covers the “last steps” and finishing touches in instructional design. This week’s article is about the beginning steps. In fact, doing a thorough evaluation of a course will be futile if the course wasn’t built right in the first place. I noted last week that evaluation methods assume a good analysis and sound objectives. This week I’d like to share with you some of the things I’ve learned about providing that good foundation, and some resources that you may want to add to your library.
Systematic design
Classic Instructional Systems Design (ISD) involves a number of steps, most of which take place before any learning materials are actually produced. (See Figure 1.) While these steps need not be the most time-consuming items to complete, their number, and the relative lack of much organized information about them, tends to discourage designers and developers from undertaking the task.
FIGURE 1 Most of the steps in ISD take place before any learning materials are produced.
These steps seem to get skipped a lot! In his 1997 survey, cited in last week’s article, William Rothwell looked at how instructional designers rated the importance of various tasks and how often they actually did them. He noted that evaluation, for example, was rated fourth in importance, but only seventh in how often it was done. The situation is even more striking in the case of the up-front analysis functions. (See Table 1.)
|
Instructional designer upfront analysis work duties |
Rated importance to success |
Frequency actually performed |
|
Conducting needs assessment |
1 |
10 |
|
Writing performance objectives |
2 |
3 |
|
Determining appropriate projects |
8 |
4 |
|
Sequencing performance objectives |
9 |
6 |
|
Analyzing characteristics of the setting |
10 |
13 |
|
Assessing learner characteristics |
11 |
9 |
|
Analyzing tasks |
13 |
14 |
|
Analyzing jobs |
15 |
15 |
The significant thing about this comparison, to me, is that needs assessment, the most critical step in “knowing where you’re going” was tenth in the order of what actually gets done. The other items, all of which are also important parts of figuring out what instruction is needed and how best to go about it, are pretty much a jumble.
One suspects the situation is about the same today among e-Learning design teams, although there is not a lot of direct evidence about this. Among the polls conducted by The Guild in the last two years, four have returned information that hints at the situation.
- In March, 2002, just over half (51.72%) said that when it comes to e-Learning, the ISD model is tried and true and useful most of the time. The rest either said it was useful “sometimes” or that it is too out of date to be useful.
- In April, 2002, a little over 54% of those who responded to the “Design Considerations Poll” said that they thought “Determining the audience, purpose, and constraints of the e-Learning program” was most likely to ensure success.
- In July, 2003, only 3.64% of those who participated felt that authoring tools could be used by Subject Matter Experts (SMEs) without any knowledge of instructional design, and the other respondents were evenly divided between thinking that it depends on the tool and the SME, and thinking that SMEs need guidance from an instructional design professional.
- In September, 2003, 21.33% of the respondents to the “Hot e-Learning Issues Poll” indicated that “Tying e-Learning to the business strategy” was the hottest issue in their organizations. This was the second-most frequent answer — most frequent was “Managing the time needed to develop e-Learning” (28.44%).
The Guild hasn’t yet done a survey that reports results directly in this area. The poll results suggest to me that there is some confusion and doubt in the community about the overall value of ISD, although there is at least nominal confidence in the idea that analysis up front is important.
What’s the effect of skipping these first steps in the design process? In a session presented to The Conference Board of Canada last April, Michael Bloom noted several underlying problems with e-Learning. Two of the major ones are that e-Learning efforts are not addressing fundamental business objectives, and that e-Learning is being approached as a technical solution rather than as a learning and business solution. A survey conducted by The Conference Board in 2003 indicated that “e-Learning efforts are not usually aligned to key organizational objectives and are poorly evaluated.” Dr. Bloom’s feeling is that this limits effectiveness and uptake. (Michael Bloom is Director, Education and Learning, The Conference Board of Canada.)
The more closely you align your e-Learning efforts to your organizational goals, the more value you deliver. And the point at which this alignment takes place is in the critical first steps of the instructional design process, which are all about value. The challenge, it seems to me, is often in the way these steps are explained, and the confusion of the various assessment and analysis procedures prescribed in popular books on instructional design. What follows is my attempt to organize this information somewhat into a sequence. Briefly, this sequence is:
- Find opportunities to deliver value — value is whatever supports the organization’s vision and strategy. But there are opportunities of different kinds: some you find, some you make, and some opportunities find you.
- Identify the most valued outcome. This usually involves defining the problem that created the opportunity. At that point, you can ask questions such as, “How much is this costing us if we don’t fix it?” or, “Will this help us to reach another valued result?”
- Figure out what’s keeping you from obtaining the valued outcome. This will usually be some combination of factors, possibly including skill or knowledge deficiencies.
- For the skill and knowledge problems, break down the learning objectives, from top level to bottom, that a learner must meet.
- For each of the learning objectives, identify the kind of performance that is involved and the alternatives available to teach this performance.
Find opportunities to deliver value
Cost reduction is not necessarily the first thing to think of when attaching a value to what you do.
Think about value before doing development the same way you will analyze value after the program is delivered. “Thinking about value” is a somewhat separate issue from instructional design itself. Purely in learning terms, value won’t necessarily affect design — but it does affect the choice of where to apply corporate resources for best advantage.
There is good reason for Guild members to identify the connection of e-Learning to corporate strategy as a hot issue in their organizations. Thinking about value helps to answer the e-Learning manager’s opening-day question, “What should we do first?” Many times advisors will suggest “start small” or “start with a highly visible issue.” My answer as stated before is, “Start with whatever keeps the CEO awake at night.” Find a way to respond to that and your worries about budget, staff, executive support and priority may be over.
In an established e-Learning enterprise, value is an important leitmotif or theme throughout the early stages of the ISD model. Value shapes the selection and implementation of a solution from the alternatives identified during the analysis steps. Getting the biggest bang for the buck is an important criterion. Whatever you choose will form the basis for evaluation and Return on Investment analysis after delivery.
Strategic focus on value: the balanced scorecard
The balanced scorecard is a strategic management approach developed in the early 1990’s by Dr. Robert Kaplan of the Harvard Business School and Dr. David Norton. The purpose of the balanced scorecard is to guide and measure the success of organizations in a way that balances the financial perspective. To do this, a company identifies a small number of broad strategic goals to define their vision and strategy. Objectives, measures, targets, and initiatives are set in four areas, to answer four key questions:
- Customers: “To achieve our vision, how should we appear to our customers?”
- Financial: “To succeed financially, how should we appear to our shareholders?”
- Internal Business Processes: “To satisfy our shareholders and customers, what business processes must we excel at?”
- Learning and Growth: “To achieve our vision, how will we sustain our ability to change and improve?”
Metrics for these four areas must tie to the broad strategic goals. According to Dr. Norton, in organizations that have been most successful with the balanced scorecard, there are usually 23 to 25 measures, 80% of which are non-financial. The objectives tie in to strategy and impact.
One excellent way to tie e-Learning to corporate strategy, therefore, would be to relate efforts and programs to the balanced scorecard categories. For example, in an international package delivery company, there is likely to be a Learning and Growth objective relating to alignment of ground crews with company goals. This in turn supports an Internal Business Process objective for fast ground turnaround, and Customer objectives for on time delivery and low prices. All of this will support Financial objectives for lower costs, increased revenue and profitability. An e-Learning initiative that supports well trained ground crews means faster turnaround, better on-time performance and higher customer satisfaction, which lead, in turn, to achievement of the cost, revenue and profitability objectives. Similar connections can be made for many e-Learning efforts in many organizations — even if the organization itself is not using the balanced scorecard.
There are a number of online resources where you can obtain more information about the balanced scorecard. One of the best for government and not-for-profit agencies is the website for The Balanced Scorecard Institute at http://www.balancedscorecard.org. Carter McNamara has assembled a page of helpful links for all organizations at http://www.mapnp.org/library/org_perf/bal_card.htm. There is also a very complete Frequently Asked Questions (FAQ) page at http://www.2gc.co.uk/resourcesfaqs.asp#faq1b.
Finally, a resource at http://www.scorecardforskills.com has been created by The Conference Board of Canada for the U.S. Department of Education. This site helps organizations tie their workplace education efforts to organizational objectives in a balanced scorecard framework. It offers a number of free services, including benchmarking against best practices.
Michael Bloom points out that, “By aligning e-Learning efforts to the things that get measured in these four key areas, you can make e-Learning central to your organization’s efforts to achieve its objectives.” That sounds to me like a good place to start.
Other ways to see value
The consulting firm Bain & Company reported that in 2001, about 36% of global companies were using the balanced scorecard. Not every organization uses the balanced scorecard. Even though you can independently apply the ideas behind the balanced scorecard in establishing value, you still need to express value in terms that make sense where you are. Here are my tips, some obvious, some perhaps not:
- The best numeric measures to have an impact on include sales, profits, and productivity (the ability to produce more without adding staff or expense).
- Many companies world-wide will be interested in the ways in which e-Learning can support management development and succession planning. In view of the coming shortages of skilled workers in North America and
- Other positive non-financial benefits to target will relate to regulatory compliance (OSHA and Sarbanes-Oxley in the
- Relate to corporate vision and strategy through the balanced scorecard categories: customers, internal business processes, learning and growth, and financial.
- Identify expensive problems where training will visibly impact or eliminate the problem and the associated costs, and will do so more effectively than competing solutions.
