In today's e-Learning world, it seems we suffer from an embarrassment of riches. Navigating the maze of e-Learning technologies, and keeping up with the ever-changing marketplace can prove a daunting task, if not a full-time job.
Editor’s Note: Parts of this article may not format well on smartphones and smaller mobile devices. We recommend viewing on larger screens.
Choosing learning technologies can be a tedious undertaking. This abundance of choices makes the job of managing learning today especially difficult. (It wasn't always this way: See Sidebar 1 for a little historical perspective.)
Not only is the selection process difficult, the consequences for choosing a system or tool that is inappropriate for your business can be costly. To do it right takes time, resources, and flexibility that are not built into the capabilities of many learning organizations.
Whether choosing sophisticated authoring software or a simple survey tool, the number of options is likely to both amaze and confuse decision makers. If an organization gets lost, and does not carefully plan the selection of a tool, the repercussions can be devastating. There is more than money on the line. There are numerous opportunity costs at risk, and in some cases, a learning organization’s reputation is also in jeopardy.
In this article, I examine the costs and common mistakes made when choosing learning technologies. I will also provide a high-level approach to help organize selection efforts and avoid costly mistakes.
Many readers of this article may never have known what it is not to be overwhelmed with huge numbers of e-Learning tools, and related applications for creating and managing multimedia. The amazing thing is that e-Learning growth happened so quickly.
Ten years ago, learning technologies were just beginning to establish their presence in the marketplace. E-Learning was a new field, with only a hint of the potential that we know it has today. Even though technology that supports learning has been around since the 1950’s, the practice had a hard time making its way in the mainstream, because practitioners and management viewed it as costly and cumbersome. It was not until the Internet became available outside the education and government spheres that innovation and innovators in the field began to emerge at a rapid pace. By 1998, the term “e-Learning” had entered everyday language, and the burgeoning field began to boom.
In 1998 however, e-Learning was not well-known among those in the training profession. By today’s standards, e-Learning was not well-supported either. Relatively few tools and vendors existed to support the market. At the time, training departments considered using technology to manage and deliver learning to be a nice-to-have, but not a necessity. Today, a short decade later, learning technologies play an integral role in the successful management and delivery of training, and most practitioners consider them an integral part of a learning organization’s success.
The paradigm shift that brought e-Learning to the forefront of many organizations’ learning strategies also brought with it a flood of new products, ideas, and services. The reach of e-Learning, and its practice, has expanded at a staggering pace since the 1990s. Where a few dozen vendors served the space ten years ago, currently there are hundreds. Recognizing the potential of the market, huge numbers of enterprising companies and individuals jumped in. The e-Learning industry today is worth billions, and serves everyone from Fortune 500 companies across the globe, to the most revered institutes of higher learning and government.
The cost of the wrong solution
Let me begin by illustrating the cost of making the wrong choice with an example of an e-Learning tool that experiences high rates of selection failure, and which often does not meet the perceived needs of its customers. Learning management systems (LMS) are among the most costly tools in terms of initial investment and ongoing maintenance. Making the wrong choice will hurt financially.
One of the most important resources for managing learning, the LMS experiences the greatest frequency of customer dissatisfaction and turnover. In fact, in the Introduction to the May 2008 Learning Management Systems 360° Report, Steve Wexler (The eLearning Guild’s Director of Research and Emerging Technologies) notes, “LMSs score some of the lowest satisfaction scores we've seen in any report.” Looking at up-to-the-minute Direct Data Access results from Guild Research on LMS products, reflecting the experience (as of last Wednesday) of several hundred organizations, I found that:
- Only 29% of organizations report receiving “very good” return on investment (ROI) from their LMS;
- 42% find their LMS to be an impediment to learning; and
- Only 31% are planning no change to their LMS in the coming year (10% are abandoning their LMS, 21% are consolidating their multiple LMSs, and 25% are upgrading their LMS).
For a variety of reasons, many organizations find themselves in the undesirable position of having selected a LMS, and realizing later that it does not meet their needs. This is typically a very costly mistake. I will illustrate just how costly with a couple of examples.
ABC Company has 5,000 employees, and it acquired a LMS to further its learning practice. (See the upper table in Table 1.) The implementation cost $428,400, and after one year, the vendor required ABC to pay an additional $223,100 for maintenance. (This example assumes the cost to purchase the LMS is $85.68 per user for installation, and $44.62 per user for maintenance. These averages were reported in The e-Learning Guild's May, 2008 Learning Management Systems 360° Report.)
|
Cost of Implementation & One Year Maintenance |
|
|
Number of Users |
5,000 |
|
Implementation Cost |
$428,400 |
|
Maintenance Cost |
$223,100 |
|
|
$651,500 |
|
Cost of New LMS Implementation |
|
|
Number of Users |
5,000 |
|
Implementation Cost |
$350,000 |
|
|
|
|
Total Cost |
$1,001,500 |
During the first year, the LMS did not deliver the expected value and the organization began asking more of the system, over and above what it could deliver. The company decided that the solution was not going to work long term, and that it needed to replace it with a LMS that could accommodate business needs. So, after an investment of $651,500, this organization is starting over.
This time, let’s assume that the company has chosen a less-costly solution at $70.00 per user, and that the total implementation investment for it is $350,000. In the span of two years, this company will have spent slightly more than a million dollars of their budget on the LMS , and that is just the hard cost. But there's more…
Table 2 shows an additional $100,800 resource cost for the implementation of both systems. There are many other costs that can come into play that I have not listed, such as hardware costs for non-hosted solutions, migrating history from one LMS to the next, and certainly the very intangible cost that comes along with not delivering what the business needs, and spending over $1,000,000 in the process.
|
Resource |
# Resources |
Hourly Cost |
Total Hours |
Cost |
|
FTE – Non-Manager |
2 |
$50 |
640 |
$32,000 |
|
FTE – Technology |
3 |
$60 |
960 |
$57,600 |
|
FTE – Manager |
1 |
$35 |
320 |
$11,200 |
|
|
Total Cost |
$100,800 |
||
LMS is not the only technology that can bring a well-intended e-Learning strategy to its knees, however. There are pitfalls associated with many of the tools we use. It is important to recognize how and why things go wrong, in order to avoid financial mistakes.
So, why is this task difficult? Why is making the wrong choice such a real possibility? The reasons abound, and are more common and mundane than you may think. They range from poor planning to slick sales pitches, and all of them are detrimental. Here are my “Top Ten” mistakes organizations make when selecting learning technologies.
Top 10 mistakes made when selecting learning technologies
In my experience, and that of others, these are the mistakes that are most likely to cause your technology selection to fail:
- Lack of strategy
- Lack of requirements
- Scope creep
- Distracters
- Lack of IT engagement
- Unrealistic expectations
- Not planning for the “nuts and bolts”
- Lingo Bingo
- Change management
- No budget, no limit
Lack of strategy
Sometimes organizations see e-Learning tools only as a means to an end, and not as part of an overall strategy. If the organization has not clearly defined the reason for purchasing a tool or system, or if a good understanding of what the solution will do and why you need it does not exist, failure is likely. You need to be able to answer why and how this tool is going to help, and what business or performance problem it is going to solve. At a minimum, understand what change you expect as an outcome of implementing the solution. If the answers to these questions are not clear, stop and rethink the perceived need. Thoroughly analyze the business, the solution’s customers, the gaps in performance, and the organization’s capabilities, in order to come up with a sound strategy.
Lack of requirements
So many times during the selection of a tool, due to lack of understanding, of time, or of resources, companies do not specifically and thoroughly define their requirements. The purpose of putting an RFP process in place is to ensure that the vendor will meet all needs. Do not pass through this important step lightly. The RFP document sets the stage for success, and tells the vendor exactly what they must deliver in order to win the business from you. If the RFP is missing steps, or if you have not thoroughly considered this process, vendors tend to guide your decision-making, and this can be costly.
The other danger here is that without detailed documentation of requirements, the buyer can miss critical features, or purchase unneeded features. A rule of thumb is to involve key customers, stakeholders, and technology partners in defining the requirements for the chosen solution. If expertise inside the organization does not exist to define these requirements, engage outside consultants, or conduct benchmarking studies. Ask questions of existing customers of a solution, or acquire industry reports to get a feel for the features and functions a solution offers. Vendors can also provide detailed checklists of functionality that buyers can use to define and prioritize requirements.
Scope creep
Scope creep can come in many forms, and can be the most common source of lag in the selection and implementation process. From business partners coming in late in the game, and adding to the list of requirements, to IT not being prepared to deliver the solution, there are an endless number of reasons for scope to change on a project. In some cases, scope creep is driven by needs or requirements that a tool or system is not yet ready to deliver. This can do two things to implementation: Slow it down by forcing you to wait for the next release of the software, or add time and cost to the implementation as the vendor customizes a solution. If a tool requires heavy customization, timing and expense will not be on your side. This setback ties to the critical step of defining requirements, and selecting a tool that can most closely deliver them.
Managing scope creep is all about engaging the right partners, and being able to articulate and document requirements. Also, clearly define what is in scope, along with what is not. This can prevent discussions and sales presentations from going down the wrong path, and can provide additional frameworks to help steer decision-making.
Distracters
Bells and whistles are nice. We select many tools because of the extras they have to offer. However, with e-Learning software more is not always better. According to Guild Research, only 18% of those surveyed use 100% of the potential of their LMS. Like heated seats on a car in Florida, some features are just not necessary, and it is your job to make that determination.
A common reason for buying more than you need is bundling. Vendors bundle products to reduce the price, and make the overall package more affordable and attractive. Let’s say you are in the market for a graphic design and editing program to augment the team’s asynchronous design efforts. The sales pitch may make it seem sensible to purchase an entire suite of software, including desktop publishing, Web development, and powerful illustration tools. The sales angle is that it will be cheaper to purchase the entire suite, and that is true, if you need all of the tools. If your staff has the skills and design savvy to use each program, this may be a good next step. If not, the suite will most likely remain unused. Let requirements drive the sales process. While bundling products can be cost-efficient, if the organization is not capable or ready to use the tools, stay on the straight and narrow, and purchase only what you need.
Lack of IT engagement
This one speaks for itself. Not having technology partners at the table, from the beginning of the process, can cause a number of problems. In fact, in Guild Research, 60% of those responding to surveys indicate that lack of IT support is a strong barrier to success. The IT group may have constraints that you don’t know about that can prevent successful installation or implementation of a solution. For instance, working with a hosted solution can prove difficult, if not impossible, if security and firewall policies across the organization restrict such activity. Bandwidth restrictions may stymie portal plans. There are a number of unknowns and risks created by not having committed technology partners. Having IT around can also provide an outsider’s scrutiny of the selection process, which can prove very beneficial. Often, these professionals are more familiar with the pitfalls associated with certain types of systems, and can help steer selection in the right direction.
Unrealistic expectations
Great expectations are good, and are a critical driver in the sustained success of a project. Unrealistic expectations are not. Software is just that, nothing more. It does not think or reason, plan or manage. Just purchasing a sophisticated authoring tool does not mean that your design staff will turn into proficient producers of e-Learning. Actually, that most certainly will not happen. You need to lay the groundwork, and the appropriate training and practice involved, to successfully use most of these tools. A LMS will not create a culture for learning simply by being available. These are merely tools, and you drive their success. Understand what these technologies do, and do not, deliver. Most important, uncover what it takes to make them a success.
Not planning for the nuts and bolts
You absolutely must consider and plan for the resources that will administer and manage this effort. So many times implementing technology correctly can be a full-time job. Even with the right team in place, the administrative end of managing technologies like LMS, KMS, portals, and some evaluation software can be extremely cumbersome.
A KMS does not organize itself, and neatly store accessible data for everyone who needs it, all on its own. A Portal cannot create, organize, and maintain communities of practice, user groups, networking activities, and knowledge sharing on its own.
Management of the majority of e-Learning and knowledge management solutions is contingent on a person or team. There is a tremendous amount of work that goes into the front end, and then continued maintenance of these systems. If not planned for appropriately (i.e. staffing changes, training, etc.) the solution will suffer, and in the worst case, end up underutilized or back on the shelf.
”Lingo Bingo”
Educate yourself, and your staff, on all the technical terms and lingo involved with the type of technology you are purchasing. In particular, the time to focus on making sure you understand all the jargon is during the sales pitch. The project leader and team will make decisions based on aspects of this terminology, so it is important to either learn it, or ask for help. From applets to ASPs, and WAP to Weblogs, there is a veritable alphabet soup of lingo to describe aspects and operating features of technology. The good news is that there are exceptional resources on the Web that can help de-mystify and translate these terms. IT partners can also play a critical role in helping understand what this language means in the context of the project.
Change management
Introducing a learning technology means introducing change, and many times organizations overlook the change management aspect of these efforts. A strong communication plan is a must, in the rollout of any enterprise system. Being able to articulate the value proposition of the tool is also mandatory, whether the system affects just the training staff, or the entire company. Additionally, make sure to involve all key stakeholders in as many decisions as possible. This strategy alone can create much-needed support for the team’s efforts.
Often overlooked, training staff can be the most affected by the change. In the early days of e-Learning, adoption was difficult due in part by the roadblocks put in place by our own colleagues. Delivery staff worried that their jobs would become obsolete; change challenged designers to learn technology that they had no interest or aptitude in. E-Learning is certainly more accepted now than it was then, but there is still a need within the learning organization, and across the enterprise, for a strong change-management strategy to be in place. As a rule of thumb, technology can be scary for anybody, so plan for good training and communication for customers of the solution. This can help avoid blindsiding anyone, and help the primary audience feel prepared.
No budget, no limit
Having a budget and sticking to it may seem like a simple concept. However, getting more than you bargain for can very well happen without a clear and limited budget. The vendor does not need to know the project budget, but you most certainly do. Even if the sky is the limit, it is still advisable to have one in place to prevent unneeded acquisitions. If the team is not strong in budgeting and negotiating, many organizations provide help from PMO (Project Management Office) or CFO (Chief Financial Officer) centers.
Additionally, remember that many e-Learning solutions require an ongoing budget allocation – most are not one-time expenses. From maintenance to upgrades, the budget will need to continue to allocate funds to the technology. Understand what those requirements are, and when they will hit the bottom line. At the end of the day, management will ask you to show ROI or ROE on this investment, so plan well, be frugal, and be tough.
