Research: What Is the Learning Effect of a Course Map on Advanced Learners?

Cognitivism uses the metaphor of the mind as a computer: information comes in, is processed, and leads to certain outcomes. Multiple cognitive learning theories describe three types of memory: sensory, working, and long-term.

Sensory memory momentarily retains impressions of sensory information after the original stimulus has ceased. It refers the unprocessed information detected by the sensory receptors and feeds it into working memory.

Working memory is where perception takes place. The memory almost simultaneously assesses the information for high emotional recognition (Can it hurt or do I like it?) and rationality (Does it make sense?). As described by Wolfe (2001), working memory, “integrates current perceptual information with stored knowledge, and to consciously manipulate the information (think about, talk about, and rehearse it) well enough to ensure its storage in long-term memory.” Information remains in working memory for about 15-20 seconds unless acted upon (rehearsed). Working memory is like the Panama Canal of learning; the vast body of the World’s knowledge on one side, a veritable limitless amount of mental storage capacity (long-term memory) on the other, and a narrow, limited gate in between. We call the capacity of working memory cognitive load, and it is cognitive load that is of concern to this study.

Long-term memory is memory that can last as little as a few days or as long as decades. It is theorized that new perceived information is more difficult to get into long-term memory than information that has some connection to previous knowledge. The value of “advanced organizers” is accepted as a design strategy to aid in learning new information by providing the lacking organization structure. A course map is a type of advanced organizer that shows the structure of the information to learn.

Suggested Further Research

Clark & Mayer (2008) suggested that what we don’t know about course navigation is “how navigational elements, such as course maps and lesson menus, should be displayed. Is there an advantage to having them always visible, as in a left navigational display …” Further, Shapiro (2005) suggests “little work has focused on site map use among more advanced learners …” It was the intent of this study to answer the questions of whether or not displaying a course map continuously to advanced learners will affect learning. We hypothesized several possible outcomes – the continuously visible map will supply or augment the necessary structure in long-term memory and aid learning; the continuously visible map will add to the working memory as an extraneous load and interfere with learning; or there may be no appreciable affect.

Study Process

Method – Post-test-only Control Group Design

Prior Participant Knowledge

Because there is no generally acknowledged definition for what is an advanced learner, for the purposes of this research, it means a learner with previous education or training in the subject area that will provide a scaffold or schemata with which to perceive and connect the newly presented information. For example, within the fall-protection training topic, the concept of an anchor point (the point that a fall-arrest system connects to) has already been learned and used in practice.

As a prerequisite to completing the study’s CBT training course, each learner has completed seven hours of training, successfully completed an examination, and demonstrated the ability to apply what they learned during a practical exercise using various types of fall protection equipment.

The Independent Variable

Employees that use personal fall-arrest systems required additional training due to changes in the Hanford Site Fall Protection Program. The training included new concepts, familiarization with a new permit, newly defined roles and responsibilities, equipment terminology, worked examples of concept applications, application exercises with feedback, and a knowledge check.

The course was broken into five required sections with an optional section containing subject-matter expert’s contact information and a knowledge check section. The only difference in the training content was whether or not a course map appeared in the upper left hand corner of each content screen (see Figures 1 and 2).

Figure 1. Site Map Continually Displayed on Screen

Figure 2. Screen Content without Site Map

Dependent variables

After completing the five required sections, the learners were required to complete a ten-question knowledge check. We used the same ten questions for the post-test, although the questions were asked in a random order. We used the learner’s performance on the knowledge check as a dependent variable.

Time to complete the course was also tracked.

Randomness

We made the CBT training course available via an intranet. Learners could take the training at their own workstation or at a central location at a time of their choosing. Using a random number generator, we assigned learners the screens containing the course map when it produced an even integer. This created an experimental group that took the training with a course map continually visible.

We launched the course on July 13, 2011, with data collected on 846 learners through September 23, 2011. During that period, 410 learners received the training with the course map continually visible and 436 learners received the training with the course map hidden until a section was completed.

Course Content Presentation and Methodologies

Due to the variability of eLearning course design, I’m providing the following information and methodology descriptions to aid you in being able to, at least to some degree, assess the quality of the training.

We based the overall instructional design on the “Nine Events of Instruction” developed by Robert Gagné, et al. (1992) and incorporated the “Multimedia Principles” advocated by Clark & Mayer (2008A). We divided the fall-protection course into six sections and a knowledge check.

Screen content began by applying the Signaling Principle (Atkinson & Mayer 2004) with a headline that explains the main idea of the screen content. Content may contain “hot words” that open a window containing a definition when the word is clicked, or hyperlinks to supporting documentation. The content included graphic organizers representing organizational and process content as suggest by Clark & Mayer (2008B).

Application exercises were included that required mathematical computations. We made a calculator available via a fly-out tab and provided feedback based on the learner’s input. The feedback to the learners’ correct answers applied the “Reflection Principle” (Colvin Clark, Ruth & Mayer R. E. 2008) by requiring learners to select an explanation for why their response was correct (Figure 3).

Figure 3. Calculation Exercise Demonstrating the Reflection Principle

Evaluation

We used a knowledge check to evaluate the learner’s mastery of the course content. A knowledge check is different from a test in that each question’s answer is graded individually and if found incorrect, the learner receives immediate remediation and must get the question correct prior to proceeding to the next question. We only gather information regarding performance on knowledge check questions for each learner’s first attempt at a question. We developed knowledge-check items with an anticipated high degree of difficulty and discrimination. For statistical purposes, we graded the learner’s performance as a standard test using the results for the first attempt for each of the ten questions that made up the knowledge check.

Results

We calculated the means for both groups for the knowledge checks and the time it took to complete the course (see Table 1), and then compared the means and calculated a “t” value. The “t” result between the two groups on time to complete the course is -0.64 and the “t” result for achievement on the knowledge check performance is -0.98. Both “t” results indicated no significant difference between completions times or content mastery.

Table 1. Calculation of Performance Means and t

Conclusions and Discussion

The intent of conducting this study was to help clarify some of the questions regarding the use of course navigational maps; how they should be displayed and how they affect the achievement of advanced learners. Based upon the results of this study, for the advanced learner there is no significant difference in achievement whether you show the course navigation continually or not, and no significant difference in the amount of time to complete the course.

We might attribute the lack of benefit from a continually visible course map to the existence of advanced learners having a scaffold or schemata within long-term memory that aided the learning of new information.

It is likely the learners, after exposure to the continually visible course map began to ignore it, and therefore it did not interfere with learning.

Since we found no significant difference, what does this mean to designers of computer-based courses? For current courses, it means you should not use time and resources to navigationally alter existing courses one way or the other for advanced learners. For future course development and design, excluding a course map would allow the space it consumes to be used to enhance learning and not affect achievement or the amount of time to complete a course.

Consideration for Further Research

One of the factors that affect our ability to recall information is time. All the learners in this study were exposed to the topic area via training or due to the nature of their work within two years of this study. There may be a point were learners will benefit from a course map if enough time has transpired to allow the long-term memory support to be forgotten. What we don’t know is at what point this may happen or how to determine if it has.

References

Atkinson, C. & Mayer, R.E. (2004). Five ways to reduce PowerPoint overload. Creative Commons. http://www.sociablemedia.com/PDF/atkinson_mayer_powerpoint_4_23_04.pdf. Assessed March 2004.

Colvin Clark, R. & Mayer, R. E. (2008A). e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning, 2^nd Ed. San Francisco: Pfeiffer.

Colvin Clark, R. & Mayer, R. E. (2008B). Learning by viewing versus learning by doing: Evidence-Based Guidelines for principled learning environments. Performance Improvement, 47 (9).

Gagne, R., Briggs, L., Wager, W. (1992). Principles of instructional design (4th Ed.). Orlando, FL: HBJ.

Shapiro, A. (2005). The Site Map Principle in Multimedia Learning. In R. E. Mayer (Ed.) The Cambridge Handbook of Multimedia Learning (pp. 313-324). New York: Cambridge University Press.

Wolfe, P. (2001). Brain Matters: Translating Research into Classroom Practice. Alexandria, Va.: Association for Supervision and Curriculum Development.

Comments

Login or subscribe to comment