Technology allows us to do some amazing things with learning. Today, everyone wants to mobilize his or her content. It’s almost impossible to attend a learning and performance event without feeling inundated with presentations on mobile strategy, or the latest in mobile authoring solutions.
The challenge remains … is the effort worth it? Most stakeholders can’t explain (at least in concrete terms) why they need to create mobile learning for their audience, let alone why they need a plan to track and report on those efforts. In order to get the most out of your mobile learning strategy, here are some questions to get you started on reporting and analyzing this content.
What are you tracking?
To begin with, you have to determine what information you will need to track. As with all training, there are standard components you want to track. Those may include course attempts, completions, scores, etc. However, mobile learning opens up a number of other opportunities. For example, how do you account for level of interaction, reuse as performance support, and social learning?
Informal and/or social-based learning is rapidly becoming a mainstay of all learning activities. Some research suggests informal learning may account for as much as 70 percent of all corporate learning. Given the nature of mobile device integration with social media (including both smartphones and tablets), informal learning will continue to take on a larger role in corporate training. It’s important to identify the specific touch points within social learning that we need to track and report on. In addition to traditional learning, we need to give focus to frequency of interactions, type of interactions, and level of contribution.
How do you track it?
Tracking learning events has long been measured around AICC and SCORM standards. Most major learning platforms and authoring tools tout their support for these standards. However when it comes to mobile learning, you need to make sure your chosen platform supports these standards within a mobile context.
The first step is to review your authoring solution. If you’re creating your own content, there are a number of solutions that claim to author for mobile consumption. Test several out and pilot them with a variety of end users. Keep in mind that tools leveraging plugins are problematic for mobile devices, particularly as pertaining to tracking and reporting. The key is to focus on native HTML- and JavaScript-based authoring tools.
Next, confirm that your method of deploying content supports tracking of mobile assets. As with the content tools, make sure the system avoids the use of plug-in-based architectures for their SCORM or AICC packages. For example, it’s common for a LMS to leverage a Java-based SCORM player. HTML- and JavaScript-based content players are generally the best solution as they are available as standard browser features without the need for a plugin.
As previously mentioned, mobile devices naturally facilitate the use of social and informal learning. You will want to track these events on a different level than you do traditional classroom or online learning as learners access blogs, wikis, Twitter, YouTube, etc. from their mobile devices. Assuming these resources are part of your learning strategy (and they should be), you will want to track and evaluate how people are using them. It’s common to find authoring and talent platform solutions integrated with these informal resources. Just make sure you include tracking and reporting on the use of those resources and that mobile devices support them.
How are you evaluating it?
You need to prepare in advance for how you’re going to evaluate your mobile learning outcomes. It’s a little too late once you start collecting the information. You must start by clearly defining the objectives and intended outcomes.
Some groups find it challenging to nail down their exact intent for mobile learning. Determining if it will be a primary method for delivering training, or whether you let it focus on training and performance reinforcement provides the bulk of the needed direction. Evaluating the impact of performance support materials is very different from evaluating that of a complete training program. With clearly defined objectives, stakeholders are able to better evaluate the effectiveness.
From an overall evaluation perspective, mobile learning remains very similar to that of classroom or eLearning. You can leverage Kirkpatrick’s Model of training evaluation as the analysis basis for your mobile content. Of course, the focus still remains on the ability to impact behavior (Level 3) and business results (Level 4). Ultimately, senior stakeholders may even ask what the ROI is.
ROI within the learning space results in a mixed response. Kirkpatrick suggests you should instead focus on Return on Expectations (ROE). This is easy to do if those expectations are set from the outset. You’re then able to match the collected metrics with the defined expectations and determine to what extent they are met.
Is it worth it?
In the end, you must ask this one question: Is the effort required to create and deliver mobile learning worth it? Mobile learning is here to stay. There doesn’t appear to be an end in sight as smartphones and tablet devices become more and more pervasive. Through continual evaluation and analysis, mobile learning strategies will become an effective component of any learning strategy. It won’t be long before informal mobile learning becomes the main element of a holistic learning and performance strategy.
