Developing a commercially successful product of any kind is difficult. Given the large costs and time involved in bringing an offering to market, many manufacturers of traditional goods use sophisticated market research, needs analyses, and focus group studies to gather requirements and drive a comprehensive design process. With a detailed blueprint in hand, only then is a product put into production, released to market, and (if lucky) subsequently evaluated with customer satisfaction surveys.
E-Learning providers typically use similar approaches to develop their own learning “products.” As evidence of this, the ADDIE model (Analyze-Design-Develop-Implement-Evaluate) is one of the most popular Instructional System Design processes used today.
Yet, like the manufacturing industry, e-Learning providers frequently find that their products — the ones so carefully researched and designed — fail to provide a satisfactory user experience. Instead, 30-75% of all students fail to complete e-Learning courses, e-Learning products are treated as disposable commodities, and e-Learning providers are constantly asked to prove their worth.
But, to address this issue, rather than applying more time and effort into “specing out” a product before placing it on the assembly line, experts in the field of Usability offer an alternative: put the learner at the center of the entire design and development process.
Don Norman, one of the founders and strongest advocates of what is called user-centered design, developed principles and methodologies to apply this process to design of e-Learning products. The result is a very different way to build a course. To illustrate, let me first describe the User Experience group within our company, then show two examples of how user-centered design principles are reflected in how we design and develop products.
UNext’s user experience (UE) group
The Unext UE Group is tasked with ensuring an overall high quality user experience for our students. To accomplish this, our UE analysts work as integral members of a course development team, paying special attention to the pedagogical effectiveness and instructability of courses, as well as typical usability issues. As part of their work, UE analysts provide expertise in the early design phase of a course and conduct many tests (a saying in our company is “test early, test often!”). They contribute to decisions that impact how a student interacts with course elements and the extended learning environment (enrollment, technical support, instructor/student community, etc.). The goal is simple: help ensure a design that provides a quality total user experience.
Figures 1 and 2 show two views of our extensive user experience labs. Each testing station is equipped with a CPU, monitor, keyboard, mouse, and camera. (Figure 1) The images from the cameras and monitors are linked to a control room via a television monitor for the camera and a slave monitor that mirrors the subject’s onscreen actions. From the control room, (Figure 2) the UE analyst running the test can observe each subject and track all user actions. In addition to our controlled lab testing, we also use remote testing and field observation studies, where users access courses from their homes and offices, and we subsequently debrief them using phone interviews and onsite visits.
FIGURE 1 Each testing station is equipped with CPU, monitor, keyboard, mouse and camera.
FIGURE 2 The User Experience analyst can observe and track all user actions from the Control Room.
The test subjects recruited for our user tests are individuals with backgrounds and expectations that match our target market: they are, in fact, our target market. While subjects are paid for their effort, we carefully structure their reimbursement to avoid receiving positively skewed feedback.
We utilize a set of structured testing programs that track each course throughout the development cycle. These programs include low-fidelity tests of interactive objects (testing using paper and/or non-graphic versions of interactive media-rich objects), component tests (targeted testing of selected course elements), integration tests (iterative testing of draft versions of an entire course), pilot tests (testing a course near commercial release), and heuristic reviews (analyses of courses by UE analysts who are not members of the development team). The methodologies used in these programs vary widely. However, together they provide an authentic mechanism to observe and involve the user throughout the entire design and development process, from initial concept to product release. We know these approaches let us catch mistakes early on, when they are cheaper to fix compared to finding them late in the development process. More important, we know our courses are qualitatively different from what we would produce were we not to practice them. Most importantly, because of these approaches, our courses provide a better user experience.
An interface design story
To illustrate how iterative, user-centered design can dramatically affect (and improve!) a product, I will describe how our current course interface came to be.
UNext’s first online learning community, Cardean University, offers online business courses and an accredited M.B.A. degree to working professionals. We create our courses in association with a consortium of leading business universities: Columbia, University of Chicago, Stanford, Carnegie Mellon, and the London School of Economics. Cardean University courses employ problem-centered learning (PCL), whereby student learning is driven by learning outcomes linked to an authentic “problem” or project. Thus, instead of watching on-line lectures or reading textbooks and taking quizzes, students are placed in real world situations where — working collaboratively with fellow students and an instructor — they apply the concepts they are learning.
As delivered via a web browser, a Cardean Course is not like a traditional web site, wherein a set of information sources is knitted together through a navigation interface. Instead, a Cardean course must present an authentic “field of practice”: a contextually related set of project and learning resources available when they are relevant to the student.
In an early version of our courses, we utilized a traditional web navigation design, the “inverted-L” (Figure 3), in which course and site links are presented along the left and top of the browser window. Largely designed without involving our learners (thanks to the rush to get a product to market!), we quickly found that students frequently got lost and frustrated in our PCL-based courses.
FIGURE 3 We used a traditional “inverted-L” navigation design in early versions.\
By observing students using this interface, we began to develop a list of new requirements to be included in a “Version 2.0” design. The long list that grew out of our user studies included such ideas as “knowledge maps,” “workspaces,” “embedded community,” “multi-modal desktops,” and “awareness.” Being the savvy designers and technologists that we are, we began designing a host of onscreen features developed around each of these ideas. However, after prototyping these ideas (Figure 4), then observing real learners interacting with them, we found that, while elegant in conception, the design suffered from a bad case of feature-itis. Put simply, students were befuddled by the array of options and unfamiliar elements in front of them.
FIGURE 4 The “Version 2.0” design was elegant, but suffered “feature-itis.”
After iterating through multiple design, prototype, test, and refine cycles with our users, we were able to determine the minimal set of features needed by actual learners to have a quality learning experience. We were able to develop a design (Figure 5) that was truly learner-centered. And, as we began coding and implementing this design in our courses, we continued (and still continue) to refine our design. The result is an interface that is both elegant in its minimalism and powerful in the way it contextually provides project and learning resources to the learner as they are needed.
FIGURE 5 A truly user-centered interface evolved with a minimalist design.
Now, contrast this experience with what would have likely played out if we followed traditional manufacturing and (ADDIE) instructional design processes. First, and most obvious, we would have confidently invested large resources to execute a formally researched design. Assuming we were actually able to realize the design in a working product, we would have found — far downstream in an “evaluate” phase — that our Version 2.0 design was no better than our previous attempt. But, because of the significant investment needed to realize that design, we would have been hard pressed to do anything other than make cosmetic fixes to shore it up: the resulting interface would be decidedly driven by technology, not by our learners.
By making our learners part of the design development process, we were able to build an environment in which we could create the type of transformative learning experience made possible through the Internet.
Building development tools
To embed user-centered design techniques throughout the development process, instructional design teams must have at their disposal integrated tools that support rapid and multiple design-prototype-test cycles. It may be obvious to say, but workflow processes that require instructional designers to run a new instructional interaction or content idea through a series of functional groups before it can “exist” in a working course discourage iterative design.
Fortunately, the new set of emerging web standards (XML, XSLT, SOAP, and others) coupled with the database technologies used in content management systems, provide the tools needed to build the design, workflow, collaboration, and management capabilities needed in user-centered processes. Within our own company, we developed UAuthor, an integrated development environment that lets instructional designers, SMEs, graphic artists, and editors iteratively design, create, and test online courses. This tool lets team members take an idea for a course element and easily add it into a working course environment. Without any assembly or coding by the design team, this environment contains the complete set of navigation, presentation, and style standards used throughout our products. Furthermore, as new interaction elements become tested and refined, we are able to add them as eXtensible Learning Components (XLCs) to our XML library of reusable learning objects.
To show how we use the tool to support our user-centered design process, Figure 6 shows a course through the lens of an instructional designer who has edit privileges on the course. As you can see, the course looks identical to that of a “real” (i.e., student version) course, except now a “developer’s dashboard” and a host of “edit” links appear. For a developer to revise any portion of the course she simply navigates to where she wants to make the change, and then clicks on the “edit” link; an XML editing window next opens, allowing the developer to revise or add to the existing content (Figure 7).
FIGURE 6 The developer interface adds a “dashboard” and “edit” links.
FIGURE 7 Revisions are made with an XML editing window.
Our usability analysts use the tool’s “student” permissions role to test ongoing versions of a course with test subjects; a “snapshot” feature lets the design team create an archived version of the course against which the Issues Database is linked. Without the combination of easy “round-trip” authoring and viewing, and integrated student-testing and issues-tracking features, we would not be able to authentically involve the learner throughout our development process.
Conclusion
User-centered design principles fundamentally change the way products are developed. Making the user a central part of the design process helps ensure that we create things that do what they are supposed to do and provide a better overall user experience. Because of its relative infancy, the eLearning industry has a wonderful opportunity to “get it right,” and keep learners an integral part of the learning environments we provide for them.
