Stevan Anas headed to DevLearn 2016 with a mission: Find a way to integrate new technology into St. John Ambulance’s first aid training to boost its appeal and effectiveness. Anas was particularly interested in the possibility of using 360-degree video in eLearning.
Jennifer Coles headed to DevLearn 2016 with a different mission: Find new technologies to integrate into eLearning and ensure that Pathways Training and eLearning, a Toronto-based eLearning vendor, remained an innovative leader.
Both organizations were interested in virtual reality (VR) and curious about whether—and how—they could create scalable, budget-friendly, immersive eLearning solutions.
Coles, the senior director of learning and development at Pathways, focused on reality: “The budgets here in Canada don’t tend to be huge for learning projects,” she said. She sought to create immersive training for “not that much more than a typical eLearning module would cost.”
Anas, an electronic pedagogy and information consultant at St. John Ambulance (SJA) in Vancouver, British Columbia, is a self-described technology enthusiast who’s always looking for ways to bring new technology into education and training. He, too, faced constraints. First aid training still relies heavily on instructor-led classroom training and is resistant to change. Another limitation: “We wanted to keep this small enough that mobile devices could use it.”
A match made at DevLearn
On the final morning of the conference, Coles, “trying to squeeze in one last session” before her flight, found herself chatting with the man sitting next to her in the Using 360-Degree Video in Training session.
Anas was about to explain to yet another clueless American where Vancouver was, when he discovered that Coles was a fellow Canadian. They soon found additional common ground.
“As a not-for-profit, we didn’t have the ability to make a 360 video or immersive experience, while Pathways was looking for a potential partner,” Anas said. Pathways planned to create a demo piece to show clients how to incorporate this new technology into their eLearning; they were looking for an organization with suitable content. “We just happened to meet at the right place at the right time.”
SJA and Pathways began discussing a joint project. “Within a few weeks, our relationship blossomed,” Anas said.
SJA is part of a 900-year-old international first aid and medical services organization. A key provider of first aid and CPR training in Canada, the organization also trains volunteers to offer first aid and first response services at hockey games, concerts, and other public events.
Anas’s team at SJA came up with a couple of ideas for the immersive simulation, including a CPR simulator, and began preproduction planning with Pathways. Within a few weeks, though, they realized that the CPR content was a poor match for the technology. “The process in itself is quite linear,” Anas said. “There’s no reason to have a 360 environment. Everything is already in front of the person.”
Coles agreed. “The benefit of 360 video is that you have to be able to look around.” Besides, she added, “CPR is something you can easily replicate in a classroom environment.”
Back to the drawing board. The other idea that Anas’s team had considered was a mass-casualty response simulation, since the Vancouver SJA chapter trains hundreds of first-response volunteers.
Coles, an instructional designer, immediately saw the potential. A mass-casualty event is not easy to replicate in a classroom. “You need to have lots of people injured, you need to have lots of actors and actresses; it’s very time-consuming if you’re doing it in a classroom every time. And how do you actually simulate that panic and shock?” she said. “We thought that that was a much better example of curriculum that would make sense using virtual reality.”
Figure 1: A volunteer makeup artist prepares “victims” for the mass-casualty simulation (Jennifer Coles/Pathways Learning)
It was settled. A mass-casualty response simulation was the perfect match for their foray into 360-degree video training.
How do you storyboard non-linear content?
The simulation would be part of a blended learning solution, Coles and Anas explained. Learners would cover the process—how a first aid responder decides which patient to help first—in the classroom. They would then apply that learning in the simulator’s realistic virtual environment. The protocol calls for a first responder to visit each patient and assess their injuries. After evaluating all patients, the responder returns to the highest-priority patient to administer first aid.
Coles tackled key instructional design decisions. Reviewing classroom content would make the VR experience too long, she decided. Instead, the simulator guides learners using feedback and graphical prompts.
In a true “unlocked” VR environment, learners can go anywhere at any time, Coles explained. But, “in order for the learning to take place, we needed them [the learners] to see every single person who was injured.” Her solution starts with a “locked” environment—a short video showing all of the injured patients that all learners watch. Then, navigation is unlocked, allowing learners to move around the environment and choose which patient to work with.
Figure 2: What a learner sees depends on where they are in the simulation; they cannot always see all five patients (Jennifer Coles/Pathways Learning)
To keep the project manageable, the teams settled on five patient scenarios. Learners experience different content with each patient, and their interactions vary according to the order in which they choose patients. Learners can interact with patients in any order, so the team had to anticipate a wide variety of interactions and script potential responses. “Instead of saying why something was incorrect, we ended up with a more universal response that would steer the learner to the correct answer,” Anas said. Learners might get feedback that reminds them of the process, such as an arrow pointing out the better choice, with the reminder, “It’s better to start with the closest patient,” for instance.
Upon choosing a patient, learners are “teleported” or moved automatically to the side of a selected patient. This strategic decision significantly reduced both the cost of production and the size of the completed module. Shooting all of the video sequences required for “walking animation” would approximately “quintuple the number of video assets,” Anas said.
Even so, preproduction was significantly more complicated than for an ordinary video. “Bar none, the biggest challenge was storyboarding it,” Coles said. For typical eLearning, she said, storyboarding is linear. You teach a concept, test learners, move on to the next idea. “How do you storyboard when you have no idea where the learner is going?”
Preproduction occupied the teams for over three months. And, Coles said, “Even as we were shooting, things were changing. In the end, the product was quite different from what had been scripted out.”
Lights, camera, action!
Julio Ordonez, product programming manager at Pathways Learning, spent about a month researching cameras and editing software during preproduction. “We considered the output files. We could have used a GoPro rig with six cameras, but in the end, you would get a massive video file,” Ordonez said. He heeded advice that Coles had picked up at DevLearn, choosing recommended plug-ins and purchasing stitching software for postproduction.
Production day arrived. SJA had borrowed Vancouver’s Rogers Arena for the day, and had recruited volunteer actors and even a makeup artist who knew how to do “casualties.”
“The production is a bit more challenging because you’re shooting 360 degrees, so there’s no option to use lighting equipment or have people standing around or behind the camera,” Ordonez said. “We relied on the arena’s lighting, which was actually really good. And everyone who wasn’t in the shot—they were just hiding in the bathroom!”
The Ricoh camera that Ordonez selected has a mobile app, which made activating the camera easy, he said. Coles revealed another trick: using a jogging headband to hold the camera during shooting.
Figure 3: Mark Snook of Pathways Learning prepares to shoot 360-degree video for the mass casualty simulation (Jennifer Coles/Pathways Learning)
Putting it all together
Ordonez started postproduction as he would with any video project, editing with Adobe Premiere and using After Effects and plug-ins to add graphical and interactive elements. He scripted and recorded audio during postproduction to add “some background noises, like people screaming and things moving around,” he said.
While authoring tools exist to create VR experiences, “they’re usually for virtual tours or things like that—not related to eLearning,” Ordonez said. He created custom code to add multiple-choice questions, embed video, track learner results—and “SCORM wrap” the entire package for use in clients’ LMSs. Ordonez created two versions, the VR version that uses Google Cardboard and a desktop version that offers 360-degree views for learners using a mouse.
In testing the immersive version, the team discovered that learners didn’t know how to use the technology. “In eLearning, we know to click next, next, next; in virtual reality, you don’t know what to do. I thought that you had to walk toward the patient,” Coles said. “The first thing I did was—I walked straight into the wall of the board room.”
But with the teleporting design, all a learner has to do is focus her gaze on the location where she wants to go, and she’s ported there. That adds in a nice bit of accessibility, Coles and Ordonez point out: The simulator can be used by anyone who can move her head.
Some early users who removed their glasses to use the cardboard viewers found the text hard to read, so a user-friendly auto-zooming feature was added, Anas said, in keeping with the teams’ goal of “trying to make things common sense and usable for as many people as possible.”
Figure 4: A page from the storyboard for the tutorial labels the branching options that learners can select (Jennifer Coles/Pathways Learning)
To teach learners what to do, Ordonez added a tutorial. “You’re in the bathroom at the Rogers Arena, and the first thing you have to do is put your sight on an arrow to activate the simulation,” Coles said. “We wanted learners to understand how you navigate in VR, because we knew that, once they got into the simulation, they might get stuck.”
Immersive simulation wows learners
Coles and Anas are pleased with reaction to the simulator, which is available as an online demo. They drew a large crowd when they demonstrated the project at DevLearn 2017’s DemoFest and during a conference session.
The realistic feel of immersive training makes eLearning more effective, Coles said. She believes that this project, which demonstrates the viability of budget-friendly immersive eLearning solutions, represents the future of eLearning. “A lot of institutions aren’t there yet in terms of adopting, but I think more and more firms will embrace it in the next year or two,” she said.
Ordonez sees the lack of VR-ready eLearning authoring tools as a potential barrier. “We’ve got custom code which we can now easily layer onto any 360 video, but for people who want to do it in-house,” it’s more complicated than creating typical eLearning, he said. Coles and Ordonez are confident that tools will be developed as demand grows; they’re even considering developing an authoring tool themselves.
People like being able to use their own phones to access the simulator, Anas said, rather than having to use a VR headset. “They were really awed by the way we’re integrating the technology into the learning,” he said.
Anas encourages training professionals to take the plunge and use 360-degree video in eLearning. He said DevLearn was a great place to network and learn. “Everyone wants to share their experiences, to create better educational tools and content.”