Connect e-Learning to Business Objectives

Albert Einstein, in his book of essays Out of My Later Years, made the observation that, “Perfection of means and confusion of goals seem — in my opinion — to characterize our age.” This is just as applicable to e-Learning today as it was to politics and physics in Einstein’s time.

One way to tell how much experience someone has with e-Learning is to notice the questions they ask. Is their concern with means or with goals? New practitioners tend to ask, “Which authoring tool or LMS or LCMS should I use?” But those who have survived their first project or two have questions about the role of subject matter experts, how to deal with “issues” over who makes key decisions, or solving problems with the constraints: how can they get enough budget, time, and staff?

Managers, designers and developers with even more experience want to know how to connect their e-Learning to the important objectives of the business. Eventually they only ask about business. They have figured out that the key to success is to discover what keeps the Chief Executive Officer or their customers awake at night — and to offer e-Learning applications that align with those concerns.

The basic mission of e-Learning development, if I might offer a definition, is to provide a system of instruction and guidance to achieve a valued result through human accomplishment. The key words here are “...to achieve a valued result....” The development team has to understand this long before thinking about storyboards, ActionScript, or SCORM. That understanding makes the difference between zeros and heroes.

Last week, I gave a quick overview of the challenges in evaluation, or quality control, of e-Learning. Evaluation covers the “last steps” and finishing touches in instructional design. This week’s article is about the beginning steps. In fact, doing a thorough evaluation of a course will be futile if the course wasn’t built right in the first place. I noted last week that evaluation methods assume a good analysis and sound objectives. This week I’d like to share with you some of the things I’ve learned about providing that good foundation, and some resources that you may want to add to your library.

Systematic design

Classic Instructional Systems Design (ISD) involves a number of steps, most of which take place before any learning materials are actually produced. (See Figure 1.) While these steps need not be the most time-consuming items to complete, their number, and the relative lack of much organized information about them, tends to discourage designers and developers from undertaking the task.

FIGURE 1 Most of the steps in ISD take place before any learning materials are produced.

These steps seem to get skipped a lot! In his 1997 survey, cited in last week’s article, William Rothwell looked at how instructional designers rated the importance of various tasks and how often they actually did them. He noted that evaluation, for example, was rated fourth in importance, but only seventh in how often it was done. The situation is even more striking in the case of the up-front analysis functions. (See Table 1.)

The significant thing about this comparison, to me, is that needs assessment, the most critical step in “knowing where you’re going” was tenth in the order of what actually gets done. The other items, all of which are also important parts of figuring out what instruction is needed and how best to go about it, are pretty much a jumble.

One suspects the situation is about the same today among e-Learning design teams, although there is not a lot of direct evidence about this. Among the polls conducted by The Guild in the last two years, four have returned information that hints at the situation.

• In March, 2002, just over half (51.72%) said that when it comes to e-Learning, the ISD model is tried and true and useful most of the time. The rest either said it was useful “sometimes” or that it is too out of date to be useful.
• In April, 2002, a little over 54% of those who responded to the “Design Considerations Poll” said that they thought “Determining the audience, purpose, and constraints of the e-Learning program” was most likely to ensure success.
• In July, 2003, only 3.64% of those who participated felt that authoring tools could be used by Subject Matter Experts (SMEs) without any knowledge of instructional design, and the other respondents were evenly divided between thinking that it depends on the tool and the SME, and thinking that SMEs need guidance from an instructional design professional.
• In September, 2003, 21.33% of the respondents to the “Hot e-Learning Issues Poll” indicated that “Tying e-Learning to the business strategy” was the hottest issue in their organizations. This was the second-most frequent answer — most frequent was “Managing the time needed to develop e-Learning” (28.44%).

The Guild hasn’t yet done a survey that reports results directly in this area. The poll results suggest to me that there is some confusion and doubt in the community about the overall value of ISD, although there is at least nominal confidence in the idea that analysis up front is important.

What’s the effect of skipping these first steps in the design process? In a session presented to The Conference Board of Canada last April, Michael Bloom noted several underlying problems with e-Learning. Two of the major ones are that e-Learning efforts are not addressing fundamental business objectives, and that e-Learning is being approached as a technical solution rather than as a learning and business solution. A survey conducted by The Conference Board in 2003 indicated that “e-Learning efforts are not usually aligned to key organizational objectives and are poorly evaluated.” Dr. Bloom’s feeling is that this limits effectiveness and uptake. (Michael Bloom is Director, Education and Learning, The Conference Board of Canada.)

The more closely you align your e-Learning efforts to your organizational goals, the more value you deliver. And the point at which this alignment takes place is in the critical first steps of the instructional design process, which are all about value. The challenge, it seems to me, is often in the way these steps are explained, and the confusion of the various assessment and analysis procedures prescribed in popular books on instructional design. What follows is my attempt to organize this information somewhat into a sequence. Briefly, this sequence is:

• Find opportunities to deliver value — value is whatever supports the organization’s vision and strategy. But there are opportunities of different kinds: some you find, some you make, and some opportunities find you.
• Identify the most valued outcome. This usually involves defining the problem that created the opportunity. At that point, you can ask questions such as, “How much is this costing us if we don’t fix it?” or, “Will this help us to reach another valued result?”
• Figure out what’s keeping you from obtaining the valued outcome. This will usually be some combination of factors, possibly including skill or knowledge deficiencies.
• For the skill and knowledge problems, break down the learning objectives, from top level to bottom, that a learner must meet.
• For each of the learning objectives, identify the kind of performance that is involved and the alternatives available to teach this performance.

Find opportunities to deliver value

Cost reduction is not necessarily the first thing to think of when attaching a value to what you do.

Think about value before doing development the same way you will analyze value after the program is delivered. “Thinking about value” is a somewhat separate issue from instructional design itself. Purely in learning terms, value won’t necessarily affect design — but it does affect the choice of where to apply corporate resources for best advantage.

There is good reason for Guild members to identify the connection of e-Learning to corporate strategy as a hot issue in their organizations. Thinking about value helps to answer the e-Learning manager’s opening-day question, “What should we do first?” Many times advisors will suggest “start small” or “start with a highly visible issue.” My answer as stated before is, “Start with whatever keeps the CEO awake at night.” Find a way to respond to that and your worries about budget, staff, executive support and priority may be over.

In an established e-Learning enterprise, value is an important leitmotif or theme throughout the early stages of the ISD model. Value shapes the selection and implementation of a solution from the alternatives identified during the analysis steps. Getting the biggest bang for the buck is an important criterion. Whatever you choose will form the basis for evaluation and Return on Investment analysis after delivery.

Strategic focus on value: the balanced scorecard

The balanced scorecard is a strategic management approach developed in the early 1990’s by Dr. Robert Kaplan of the Harvard Business School and Dr. David Norton. The purpose of the balanced scorecard is to guide and measure the success of organizations in a way that balances the financial perspective. To do this, a company identifies a small number of broad strategic goals to define their vision and strategy. Objectives, measures, targets, and initiatives are set in four areas, to answer four key questions:

• Customers: “To achieve our vision, how should we appear to our customers?”
• Financial: “To succeed financially, how should we appear to our shareholders?”
• Internal Business Processes: “To satisfy our shareholders and customers, what business processes must we excel at?”
• Learning and Growth: “To achieve our vision, how will we sustain our ability to change and improve?”

Metrics for these four areas must tie to the broad strategic goals. According to Dr. Norton, in organizations that have been most successful with the balanced scorecard, there are usually 23 to 25 measures, 80% of which are non-financial. The objectives tie in to strategy and impact.

One excellent way to tie e-Learning to corporate strategy, therefore, would be to relate efforts and programs to the balanced scorecard categories. For example, in an international package delivery company, there is likely to be a Learning and Growth objective relating to alignment of ground crews with company goals. This in turn supports an Internal Business Process objective for fast ground turnaround, and Customer objectives for on time delivery and low prices. All of this will support Financial objectives for lower costs, increased revenue and profitability. An e-Learning initiative that supports well trained ground crews means faster turnaround, better on-time performance and higher customer satisfaction, which lead, in turn, to achievement of the cost, revenue and profitability objectives. Similar connections can be made for many e-Learning efforts in many organizations — even if the organization itself is not using the balanced scorecard.

There are a number of online resources where you can obtain more information about the balanced scorecard. One of the best for government and not-for-profit agencies is the website for The Balanced Scorecard Institute at http://www.balancedscorecard.org. Carter McNamara has assembled a page of helpful links for all organizations at http://www.mapnp.org/library/org_perf/bal_card.htm. There is also a very complete Frequently Asked Questions (FAQ) page at http://www.2gc.co.uk/resourcesfaqs.asp#faq1b.

Finally, a resource at http://www.scorecardforskills.com has been created by The Conference Board of Canada for the U.S. Department of Education. This site helps organizations tie their workplace education efforts to organizational objectives in a balanced scorecard framework. It offers a number of free services, including benchmarking against best practices.

Michael Bloom points out that, “By aligning e-Learning efforts to the things that get measured in these four key areas, you can make e-Learning central to your organization’s efforts to achieve its objectives.” That sounds to me like a good place to start.

Other ways to see value

The consulting firm Bain & Company reported that in 2001, about 36% of global companies were using the balanced scorecard. Not every organization uses the balanced scorecard. Even though you can independently apply the ideas behind the balanced scorecard in establishing value, you still need to express value in terms that make sense where you are. Here are my tips, some obvious, some perhaps not:

• The best numeric measures to have an impact on include sales, profits, and productivity (the ability to produce more without adding staff or expense).
• Many companies world-wide will be interested in the ways in which e-Learning can support management development and succession planning. In view of the coming shortages of skilled workers in North America and Europe, value will be created by any initiative that positively impacts recruiting and retention of these individuals.
• Other positive non-financial benefits to target will relate to regulatory compliance (OSHA and Sarbanes-Oxley in the United States, for example, or perhaps privacy or electronic commerce regulations in other countries) and customer satisfaction.
• Relate to corporate vision and strategy through the balanced scorecard categories: customers, internal business processes, learning and growth, and financial.
• Identify expensive problems where training will visibly impact or eliminate the problem and the associated costs, and will do so more effectively than competing solutions.

Find the most valued outcomes

This is the needs assessment stage, sometimes unfortunately referred to as “training needs analysis.” It’s unfortunate because calling it “training needs analysis” implies skipping ahead to decide on a solution (training, e-Learning, blending) before the problem is identified. Thinking of this step as problem identification, or as the first part of what I’ve been referring to as “up-front analysis” may help. After this step, you should know the difference in human performance between what people are doing and what they would be doing in a perfect world.

Admittedly, this sounds a bit esoteric — it’s a problem shared by most descriptions of needs assessment. It may help to remember that needs assessment was at one time a somewhat political process, used to decide what should be included in the curriculum of the public schools in the United States . You can find a description of this in Roger Kaufman’s Educational System Planning, together with a concise discussion of the ISD process (see the sidebar). If you are not familiar with the formal ISD process, I recommend this book as a useful overview, even if you work in an industry that would never tolerate the level of detail Kaufman describes.

How do you perform a needs assessment?

Needs assessment is still a slightly messy and inexact process. However, William Rothwell provides an extensive description of various options for collecting needs assessment data, under an overall needs assessment plan. His approach is certainly thorough, and resembles (as he says) a research plan. Under some circumstances this would be appropriate, but many times the team (or the individual) doing the assessment does not have the luxury of the time or the resources required for a formal needs assessment. In these cases, the needs assessment normally proceeds by using one (or more) of several approaches:

• Interviews with performers, centered around concerns expressed by managers or decision-makers;
• Direct observation of performers on the job;
• Review of production or administrative records;
• Questionnaires completed by performers; and
• Various forms of brainstorming or focus groups

Each of these has its particular advantages and drawbacks, mainly having to do with cost and time, but most can be reliable ways to identify discrepancies in concrete, measurable terms.

In the needs assessment, then, you are first going to be identifying discrepancies in concrete and measurable terms and then deciding whether these discrepancies are important. For the second part, it may be enough to ask, “What would happen if you left it alone?” If a discrepancy is not important, you can move on to the next one. Eventually you are going to have a list of the important discrepancies and you can identify the priority items.

What’s stopping you?

This is where you figure out what’s keeping you from obtaining the valued outcome. This will usually be some combination of skill or knowledge deficiency, poor motivation or inappropriate consequences, obstacles to performance, ineffective processes for selecting employees, or bad design of jobs, processes, or organizations. e-Learning can only deal with skill or knowledge deficiencies, but it’s important to be sure you aren’t, for example, trying to use training to fix a job design problem.

As Ron Zemke says in Figuring Things Out, it’s a fundamental mistake to directly challenge the idea of a decision-maker that “we need some e-Learning.” But remember that at this point, we still aren’t designing training. We are analyzing the priority needs to see which ones have skill or knowledge deficiencies as their causes.

Fortunately, Peter Pipe and Robert Mager provide an excellent blueprint for the process in Analyzing Performance Problems. The systematic approach they describe actually parallels the classic ISD model, and many practitioners over the years have found Pipe and Mager’s description more suited to their situation. It is also somewhat easier to explain performance analysis to decision-makers by using Pipe and Mager’s book. Allison Rossett’s book First Things Fast provides more information on performance analysis.

There are several other procedures that, like performance analysis, can help you decide whether the discrepancies are going to be ones that you can address with instruction or whether another approach would be a better answer. These are described in detail in Mastering the Instructional Design Process and in Developing Vocational Instruction by Robert Mager and Kenneth Beach, Jr.

• Critical incident analysis describes what happens when the performance is not correct. By identifying what isn’t going right, it should be possible to identify what can be done about it.
• Task analysis is a description of what a competent performer does, made from actual observation of the performance. This directly supports development of learning objectives.
• Target population description looks at the characteristics of the learners. This is more for the purpose of adjusting learning objectives than for figuring out what the learning objectives should be.
• Goal analysis is a special case, described in Mager’s Goal Analysis. You may sometimes have an important goal like “safety awareness,” that is not a performance but an abstraction, difficult to define in terms of exactly what is wanted. Goal analysis is a way to come up with descriptions of specific outcomes that you would agree mean that the goal is also achieved.

Breaking down the objectives

This is, I hope, familiar territory. At this point you have identified priority performance needs related to important strategic organizational goals, and you have analyzed those needs to determine what you can do about them, through instruction, to correct skill and knowledge deficiencies. In other words, it is time to set some learning objectives.

Learning objectives describe the results of instruction, rather than the process of instruction. The definitive guide to writing top-level (sometimes called “terminal”) objectives is Robert Mager’s Preparing Instructional Objectives. William Rothwell’s book also contains a complete guide to writing objectives, including cross-cultural considerations.

To me, Mager’s description of an objective has always been the clearest:

An objective is an intent communicated by a statement describing a proposed change in a learner — a statement of what the learner is to be like when he has successfully completed a learning experience. It is a description of a pattern of behavior (performance) we want the learner to be able to demonstrate.

Another of Mager’s books recommended above, Goal Analysis, addresses development of learning objectives where the outcome was initially described in “fuzzy” terms (“We want learners to appreciate the importance of safety.”).

A top-level objective may require one or more supporting, or enabling, objectives. The learner may need to be able to use a micrometer before she can machine a part to within tolerances, for example. Once the top-level objectives are defined, the enabling objectives are broken out. These hierarchical sets of objectives guide the rest of the design process.

Methods and tools

For each of the learning objectives, the next step is to identify the kind of performance that is involved and the alternatives available to teach this performance. This is a topic that could (and does) occupy entire books, so I’m only going to summarize it here. This is the point at which the designer looks at the different methods and means available to support learning for each objective. It is common to hear decisions made about whether a given need is going to be addressed by e-Learning, blending, or instructor-led methods even before the learning objectives have been defined. However, these decisions are typically best made after methods and tools analysis has been completed.

Performance types

Sometimes instructional designers choose the methods with which they are most comfortable or familiar. Logically, though, it makes more sense to choose the method that is best matched to the outcome and the kind of performance required to produce it. As Mager and Beach point out, a simulator is a great way to teach someone to fly an airplane, but a bad way to teach spelling. Two different kinds of performance are involved.

There are a couple of ways to look at the question of performance types. Mager and Beach, in Developing Vocational Instruction, suggest a system of five categories:

• Discrimination: knowing when to do something, or knowing when it is done
• Problem-Solving: deciding what to do
• Recall: knowing what to do and why to do it
• Manipulation: knowing how to do it
• Speech: knowing how to say it

Ruth Clark, in Developing Technical Training, offers a content-performance matrix that has the same basis as Mager and Beach’s categories. In this matrix, the learner will be asked to either apply or remember the information to be taught. The information can be classified as facts, concepts, processes, procedures or principles.

Review each of the learning objectives to determine the performance that is required and, in the case of Clark’s matrix, the type of information being taught.

Methods and Tools Analysis

Once the designer has identified the performances required, Clark, and Mager and Beach, suggests appropriate methods for teaching it. These are mostly media-independent.

Mager and Beach provide three guides to identifying methods or techniques for teaching the performance:

1. . Choose the technique that most closely approximates the performance conditions called for by the objective.
2. Choose the technique that causes the learner to perform in a manner most closely approximating the performance called for on the job.
3. Choose the technique that will allow the learner to make the largest number of relevant responses per unit time.

Clark offers a different approach. For each of the information types (facts, concepts, processes, procedures, principles), she has a whole chapter on how to teach it. Teaching concepts, for example, involves helping the learner remember the definition and then systematically developing the learner’s ability to apply the concept by correctly identifying examples. Clark also suggests best practices for both classroom and e-Learning delivery.

Once the most relevant or appropriate methods have been identified, advantages and disadvantages are listed for each and the designer can choose the one that is most practical or affordable. It is at this point that the designer begins to make decisions about the use of e-Learning, and about sequencing the content and blending delivery methods.

Conclusion

This has been a very fast review of resources and ideas to help e-Learning managers, designers, and developers think about the critical early steps in instructional design. This part of the design process is where the link is made between e-Learning and organizational strategy, but it is the one part most often “passed over lightly.” By beginning with a strategic focus (perhaps using a balanced scorecard approach), and then systematically identifying and analyzing priority needs, followed by identification of objectives and analysis of the appropriate methods and tools, there is a far greater chance that the finished e-Learning or blended product will provide maximum value to the organization.

REFERENCE

Bloom, Michael. “Developing a Balanced Scorecard Approach to Measure the Performance of Your e-Learning Initiatives.” Presentation made April 1, 2003 to the Conference Board of Canada. Retrieved April 1, 2004 from http://www.conferenceboard.ca/education/reports/pdfs/elearning_presentation.pdf

Comments

Login to comment

Related Articles

The 2010 Horizon Report: What Learners Look to Us to Learn

3/5/10

Technology tools that might still be on the periphery of our personal e-Learning radar screens are about to be adopted widely by those we serve, a newly released report shows. If we don’t develop an understanding of and familiarity with them now, the learners who currently turn to us for assistance may leave us behind.

Collaborative Learning – for the people, by the people

3/4/10

Traditional approaches to training are facing disruption. Disruptive innovation, in the form of social software, is sparking new philosophies about formal and informal use of collaboration to support learning. This is the first of two columns about this, and how you can adapt your approach to instruction to take advantage of these developments.

Beta Testing an Online Course

1/11/10

Beta Testing is a key step in creating insanely great e-Learning applications. Often overlooked or omitted out of concern for budget and schedule, it is actually simple and cheap to do, and when planned with care it takes little time to execute. Done before development is 100% complete, it ensures faultless releases. Here’s a step-by-step guide from an experienced developer.

You need to upgrade your Flash Player
This interactive requires Flash Player version 7 or higher.