Using xAPI to Evaluate Successful Completion of eLearning

Written By

Art Werkenthin

February 07, 2019

An interesting question arose at a recent xAPI Camp hosted by The eLearning Guild: “What happened to objectives in xAPI?” We should be able to use xAPI to document successful completion of eLearning, but without statements of learning objectives in the content, this is not possible. Let me explain the context.

I was presenting a customer case study. We had a customer that was in the midst of a serious rewrite of their learning content. Why? They were attempting to reduce their safety incidents. Students were successfully completing their eLearning content and passing tests, yet there did not seem to be any impact on safety. They were also using performance checklists, where a field supervisor would assess a person’s performance on safety-related tasks. This is where the performance gap appeared—passing a test did not seem to equal successful completion of the performance checklist.

Being xAPI geeks, we naturally started digging through the xAPI statements. We decided to chart passed vs. failed tests and eLearning content by objective. What we found was a bit disturbing. While test questions (which came from the LMS) were associated with objectives, the largest bar on our chart had “unknown” as the name of the objective. Where did this data come from?

The answer? SCORM.

I know, I know… SCORM has the ability to record objectives. But when was the last time (or ever) that you saw a place to put objectives in SCORM content developed with any popular rapid authoring tool?  More importantly, shouldn’t each test question in SCORM content be associated with one or more objectives?

“What happened to objectives?” The xAPI Camp discussion was fast and animated from that point.

Instructional systems design models and objectives

As I recall, the identification and development of objectives begins in the first phase of the ADDIE model of instructional design, Analysis. During the Design phase you create evaluation instruments to measure mastery of those objectives (this is why objectives begin with a verb).

OK, you don’t like ADDIE and you want to use an Agile approach instead. Step one is to design your learning objectives and map your interactions. Hmm, objectives… that sounds familiar.

Regardless of the ISD model you prefer, you are going to develop your objectives somewhere early in the process. Ultimately you will measure a person’s success against your learning objectives.

What went wrong?

One xAPI Camp attendee mentioned that early content development tools had a place for objectives. But current popular rapid authoring tools, for the most part, have no place for them. Another attendee summed up the best explanation for this: “PowerPoint.”

I don’t think Microsoft originally intended PowerPoint as a tool for rapid authoring of eLearning content. However, PowerPoint seemed like an easy solution when companies wanted the SME to develop inexpensive, high-volume content.

Those of you who have been around for a while remember that many popular tools for rapid authoring were originally released as PowerPoint add-ins. Suddenly you could convert your PowerPoint to Flash! Streaming! Fast! No PowerPoint files to download! Awesome!

However, no objectives.

Does xAPI fix it?

The short answer is no. Sorry. There is nothing in the xAPI specification that forces content to associate objectives with anything, but xAPI is nothing if not flexible. xAPI “allows” you to associate objectives with any statement. What we need is a way to do that consistently, like a profile…

Proposed solution

At the beginning of this article I presented a problem my company faced: We could not find objectives in a customer’s xAPI statements for their eLearning content. How could we fix this?

We started by looking at where existing xAPI content was identifying objectives. We found only one example: cmi5. While the cmi5 specification does not require adding objectives, it does have a best practice recommending how they should be recorded. In fact, it is Best Practice #1. Some of the sample statements for cmi5 show the actual structure of a statement with objectives.

cmi5 recommends the following:

  1. The contextActivities of your xAPI statement should include a “parent” element representing the objective.
  2. The “definition.type” of the Activity should be “http://adlnet.gov/exapi/activities/objective”.
  3. definition.name”should include the objective Name… (please don’t make the reporting tool guess what your objective’s ID is supposed to mean).

Below (Figure 1) is an example of a “cmi5 allowed” statement’s contextActivities referencing an objective.

cmi5 code snippet

Figure 1: How cmi5 allows contextActivities referencing an objective. From the cmi5 JSON examples page.

Objectives utopia

Imagine…

  1. On pages where your content addressed specific objectives, your content would write an xAPI statement that included the objectives covered.
  2. On pages that included a test question, the “cmi.interaction” statement sent by your content would reference the associated objective(s) for the test question.
  3. You could select a person and an objective and get a list of all activities performed by them that referenced that objective.
  4. If a person was consistently having trouble with one objective, you could intervene with coaching on that one objective only.
  5. If the same objective(s) is missed systemically by many students, it is time to evaluate the content and assessments to ensure they are aligned with the objective. 

This is how you can evaluate successful eLearning completion. Isn’t this the way ISD was supposed to work?

More Development

You May Also Like