Enterprises can receive great benefit from e-Learning and training managers have realized this. According to ASTD’s 2010 Learning Trends report, over 18% of enterprises spent more than half of their entire training budgets in 2009 for blending or substituting traditional training with e-Learning.
Nevertheless, financially evaluating corporate e-Learning investment decisions remains difficult: the popular approach in this field, Return On Investment (ROI) analysis, requires a comparison of the cost of a program with its monetary benefits. Problems arise mainly from the latter.
In this article I introduce the less complex Cost-effectiveness Analysis (CEA) for identifying economically beneficial e-Learning investments. In contrast to ROI analysis, CEA does not require the translation of training outcomes into monetary training benefits in order to produce insightful results.
I will focus on decisions regarding blending or replacing traditional training with e-Learning, where CEA is especially feasible. After highlighting key issues when assessing training cost and effectiveness, I will introduce the Cost-effectiveness Pane as a powerful tool to communicate CEA results by providing a clear visual representation of the changes in cost and effects. I close the article with advice on how to deal with multiple outcomes and a quick review of key concepts.
Costs and effectiveness
The two major economic changes for an enterprise that is substituting or blending traditional training with e-Learning are:
- Training cost changes
- Changes in training effectiveness
Training cost and effectiveness are the chief components of CEA, and being clear about these points is essential to attaining meaningful results. Here is a brief introduction to Cost Analysis (CA) and Effectiveness Analysis (EA).
Cost analysis
Depending on the nature of the face-to-face or e-Learning training, different costs may occur. Whether, for example, staff travel expenses appear in the CA of a face-to-face training depends on where the training takes place. Travel costs occur only when the course is not delivered in-house. The analysis only has to consider development for an e-Learning course if the course is specially developed for one’s own organization. If the organization bought a standard course, user license costs appear instead.
It is important to base the cost formula on fixed costs, variable costs, and the total number of participants.
The cost formula
Costs are either fixed or variable in relation to the number of participants. Fixed costs, such as course development costs, occur independently of the number of participants. This means that the total fixed cost remains the same, no matter how many people join the course. Thus, the fixed cost per person actually decreases as the number of participants increases. Economists call this phenomenon “economy of scale”. Variable costs, such as travel expenses, occur for each participant individually. This means that total variable costs increase along with the number of participants but the variable costs per person stay constant.
Therefore, we can use the following formula to calculate the total cost of both face-to-face training and e-Learning:
Total Cost = Fixed Cost + Variable Cost x No. of Participants
Irrelevant cost
We need not include in our cost calculation those costs that are the same for both traditional and e-Learning solutions (e.g. the salary of the training manager). The reason for this is that we are comparing the changes in cost of substituting or blending an existing face-to-face training with e-Learning. Including these costs for both alternatives will yield the same result but will create more work than is necessary.
When deciding whether to blend or substitute an existing face-to-face training with e-Learning, you should not include the costs which have already been paid for during the face-to-face training (sunk cost). For example, you have developed a face-to-face training for which now there are only variable costs (i.e. the trainer’s salary). These development costs are considered sunk cost and can thus be ignored. For the new e-learning alternative, however, you should include these costs.
A CA example
Let us analyze the cost of substituting an existing tailor-developed face-to-face course with that of a similarly tailor-developed e-Learning course. Figure 1 shows the total cost of both alternatives in relation to the number of participants.
Figure 1: Cost comparison of a personalized face-to-face training versus a personalized e-Learning course, dependent on the number of participants
Both courses have fixed development cost. The development cost of face-to-face training is clearly lower than that of e-Learning. On the other hand, once developed, the e-Learning course does not incur any further costs. Therefore, the total cost line for e-Learning is horizontal. The traditional course, however, contains such variable costs as the cost of the trainer. Consequently, the total cost line of the face-to-face course continues rising after development. When a particular number of participants is reached at N1, the total cost line of the face-to-face training intersects with that of the e-Learning training. So after this number is reached, the total cost of e-Learning is less than that of a traditional training. From N2 on, the additional cost of e-Learning is less than the additional cost of continuing the face-to-face training.
Effectiveness analysis
CEA compares the cost and effectiveness of different alternatives with the same goal. In the previous section we analyzed the costs of blending or substituting a face-to-face training with e-Learning. This section deals with the Effectiveness Analysis (EA) of the two alternatives.
Effectiveness is the extent to which an activity fulfils its intended objective. Finding the appropriate measure(s) of effectiveness is essential for a successful EA.
Levin and McEwan (2001) emphasize that evaluators need to consider two general concepts when choosing among measures of effectiveness: reliability and validity. A reliable measure is one that yields the same results when applied repeatedly to the same individual. A measure is said to be valid if it reflects a close correspondence to the main objective of the alternatives. The evaluator's goal is to employ the most reliable and valid measures available.
Donald Kirkpatrick is a pioneer in the science of corporate training effectiveness evaluation (Kirkpatrick 1959, 1960) and his model (Kirkpatrick 2006) is still by far the most popular one used by organizations today (Bates, 2004). It consists of four levels of training outcomes:
- Reaction: measures learners’ satisfaction with the program
- Learning: measures changes in knowledge, skills and attitudes
- Application: measures changes in job related behavior
- Results: measures changes in business impact variables
CEA can be done most easily if we focus on exactly one measure. It should be the most reliable and valid measure available, revealing to what extent both face-to-face and e-Learning fulfill the intended main training objective. For new employee induction training you could use Kirkpatrick’s Level 2 test results as an effectiveness measure.
In the next section, I will assume that there is only one such effectiveness measure. Later, I will discuss how to deal with more than one effectiveness measure.
