Faster, Better, Cheaper: Improve Corporate e-Learning Content Management with MERLOT

“Faster — better — cheaper.” Those three words have long expressed the dream of project managers, instructional designers, and corporate executives.

While none of us will ever attain the ultimate goal of instant, perfect, and free e-Learning, if we look at the words as comparative adjectives, we can always do better on these measures than we are currently doing. In this article I’ll introduce you to a system that is already supporting such improvement in the education field, and offer some thoughts on how it can work in both private enterprise and government institutions.

The Multimedia Educational Resource for Learning and Online Teaching (MERLOT) project is both a community of practice and an international cooperative of educational content available as an online, searchable collection of links to learning objects. (Please refer to sidebar 1, “A Learning Object Lexicon” for definitions of unfamiliar terms.”) Participants use the MERLOT Web site (http://www.merlot.org) as a database of reliable resources and high quality materials. Instructional designers can quickly and easily incorporate these into a course or curriculum. Secondarily, MERLOT is a place where members of the community of practice can contribute materials and receive recognition for their work.

MERLOT is not a content repository. Rather, MERLOT provides an easily searchable database of seven categories of links to content housed on the originator’s site. The database provides links to thousands of learning resources from around the world. Since MERLOT does not house the actual materials, the organization is not responsible for the content or for the upkeep of the host sites.

Today, only educational institutions use MERLOT. However MERLOT’s standards, structures, and approaches can apply to and provide value in a corporate setting. A MERLOT-based approach will work outside of education and can provide an alternative to other enterprise-wide content management solutions.

The basis for much of the content here is in the References at the end of the article. I will not cite all of them individually, but each of the authors deserves credit for his or her contributions.

The need for reusable content

I’d like to begin by asking two questions about the need for reusable content and for a database to manage such content. Can learning objects help corporations achieve operating efficiencies, increase product flexibility, reduce time to competence in performers, and increase a corporation’s competitive advantage? If so, what is the best way to manage objects in a corporation?

The case for learning objects in corporations

Knowledge leaves corporations every day, carried away in the minds of employees who retire, become redundant, or who make a career change. As Allison Rossett and Kendra Sheldon point out, knowledge “is being frittered away through carelessness and attrition.” This drain helped identify the need for a better way to define, store, and use corporate knowledge — a way to identify, capture, and disseminate best practices to employees who need the information.

Learning objects are an efficient way to categorize, contextualize, and tag this knowledge and to make the knowledge accessible in a variety of training, documentation, reference, and support venues. As Ruth Colvin Clark explains, “Learning objects provide an effective way to improve knowledge capital in an organization.”

It is expensive to store large amounts of content in traditional, hard-copy format, and storage itself consumes large amounts of space and time. To make matters worse, accessing specific information in hard-copy storage can be daunting.

Storage of learning objects is far less costly since large physical structures are not required. It is also much easier to access and display learning objects at a variety of points on a network. Learning object tags make maintenance, searching, and management simpler.

Corporations traditionally develop instructional content for single use, event-based interventions. In the last few years the amount of corporate content created as reusable learning objects has grown for a number of reasons, including:

• Reducing the cost of learning development
• Ensuring message standardization (the same information is consistently presented)
• Enabling timely customization and personalization of learning to meet specific needs

The learning object concept has parallels in many well-established business practices. For example, manufacturing processes assemble standalone components into products. Standalone components provide flexibility and allow for rapid product creation and modification to respond to market demands. In fact, several manufacturers may use the same standardized standalone components in a variety of different products. This approach will also work with an asset such as corporate knowledge.

This is the purpose of learning object design. Designers can compile objects into many different learning sequences. Furthermore, performance support systems can use the component parts of those learning objects (the “assets” — images, text, audio files, etc.) to support employees with the exact procedure needed at any given moment. This supports a larger objective, the delivery of appropriate content to performers at the moment of need, in the format required, using the most available device (Web, mobile phone, PDA). Learning objects offer the potential to help develop new products and penetrate new markets more quickly, by leveraging content, personalizing learning and communication, and streamlining learning maintenance. Some organizations are already realizing this promise.

The need for a database

As with academia, for shared resources to work in a corporation all those who are involved in a project must share a set of needs and problems that can be addressed most effectively through collaboration. Use of a common technology leverages investments made by each part of the organization, achieving greater benefit and reducing costs for the organization. The learning object database becomes a common tool for individual work groups to meet their content needs, giving each group back more than they have contributed.

For a collaborative database to work, corporate participants at every level will have to perceive that there is an equal participation and a fair share in decision-making. The database needs to be visible, accountable, and sustainable to build the level of trust required for the necessary level of participation, both in terms of submissions and in terms of use of objects.

Corporations that manage content often do so through a learning content management system (LCMS). These systems not only provide access to content but they also house the content, manage the learner interface with the content, and handle learner and learning administration activities. They are most often enterprise-wide applications that receive information feeds from the Corporation’s human resource management system. LCMSs require significant funding both to purchase and to implement and, since they link to existing corporate systems, there can be security and integration issues which extend timelines, resource requirements, and cost. MERLOT, on the other hand, is a web site implementation. The Web site simply accesses a database of links, does not hold content, and does not manage learners or learning administration. As a result, a MERLOTbased approach to content management is faster, easier to scale, and requires fewer resources to implement and administer.

To determine whether MERLOT could effectively meet a medium to large-size corporation’s needs, the challenges encountered with MERLOT in academia need to be considered and reviewed from the perspective of the corporation. I will review the challenges at the end of this article, after I briefly explain how MERLOT works. In addition, Sidebar 2 offers a real example.

How MERLOT works

MERLOT’s creators made the basic assumption that thousands of colleges and universities cover the same content. This duplication needlessly consumes time, money, and resources. This assumption is equally valid in corporations where several departments or business units each create their own version of a course without being aware of materials already developed in other organization units.

MERLOT represents a form of market economy among universities based on the reuse of interoperable, accessible, and structurally sound learning content, so that the value of the content increases with every use. These learning objects are modular and nonconsumable. The owners of the content can sell the objects to an infinite number of institutions who then house them in their own repositories as part of their own course offerings. Allowing objects to compete in the MERLOT market economy increases the availability of effective and functional learning content. It also increases the likelihood that authors are rewarded for the quality of their materials while protecting intellectual property rights. Locating suitable learning materials for a specific topic or teaching point on the Web is difficult due to the unmanageable number of links and because evaluating search results takes time. Once designers locate and evaluate a potential exercise, document, or application, the next question is whether the materials will function in the designers’ technical environment. In response to these issues, MERLOT’s structure and processes simplify access to content. MERLOT’s structure provides optimum access and flexibility while limiting the amount of central administration and responsibility for content maintenance. Designers can search for subject area, material type, technical formats, or primary audience. The system provides a list of resources, sorted so that peer-reviewed, highly rated items appear first. Content is open-source, public, and coded and tagged to work in AICC- and SCORM-compliant learning management systems.

Selecting a link in the MERLOT database opens an information page on the resource. (See Figure 1.) This page provides user comments, peer reviews, assignments, and any ratings available for the resource. Other details include information on copyright restrictions, the target audience, and the format of the digital resource. MERLOT also checks the host site to see if there are any licensing regulations or costs associated with the use of the resource. The resources are not all free, nor are they all free of restrictions. However, MERLOT is not responsible for the collection of fees or the policing of restrictions. Responsibility for content management remains with the originator, to limit the administrative burden on the central organization.

Figure 1 MERLOT Learning Resource Detail View Reproduced with permission from the MERLOT repository at http://www.merlot.org/artifact/ArtifactDetail.po?oid=1010000000000025300 (September, 2004)

Membership in MERLOT requires only the completion of an online form available on the Web site. Membership is free and anyone, whether a member or not, may access the resources. However, only members can contribute resources, search the networking directory, post comments, and create personal collections of resources. Interestingly, members can add content created by others. When someone other than the author adds content, the system advises the resource owner by email.

The MERLOT links connect to content designed and packaged as learning objects: chunks of context-free learning facilitating the reuse of content across many situations for different purposes. Learning objects are a cost-effective way to design learning or to include alternate explanations, activities, and examples in classroom based or e-Learning courses. By only linking to learning objects, MERLOT is ensuring that, by definition, materials will be reusable and transportable, and chunked and designed for ease of reuse.

Challenges

In any system that supports collaboration and sharing across different (and sometimes competing) organizations, there will be challenges to efficient functioning and even to survival of the system. MERLOT is no different. In fact, there are four important challenges that any organization must keep in mind. I will summarize these challenges here, describe how MERLOT meets them, and suggest some methods that may be useful in the corporate setting. Table 1 compares the four challenges and suggests how to address each.

Challenge One: Ensuring content quality

It is important to ensure the quality of materials linked from MERLOT, in the same way that scholarly journals closely control the quality of articles they publish in order to establish and maintain credibility. John Nesbit, Karen Belfer, and John Vargo contend that establishing learning object validity through third-party evaluations is key to promoting reuse and increasing the efficacy of searches.

To address this challenge, MERLOT established a peer review process in which qualified faculty, sitting on panels organized by related disciplines, evaluate contributions, against published standards. The process is transparent: identities of the reviewers are known and the reviews are posted. Each Disciplinary Community has an editorial review board responsible for developing evaluation standards and conducting peer reviews of materials for that discipline. There is also a Discipline Editors’ Council that oversees the activities of the Disciplinary Communities.

The discipline-based processes used in traditional scholarship and research provide the model for MERLOT’s peer review process. The process seeks to ensure that materials accessible through MERLOT are contextually accurate, pedagogically sound, and technically easy to use. Panels rate materials on a scale of 1 to 5 in each of these three categories. “This nuanced three-pronged approach to rating materials is invaluable when it comes to choosing materials for a concrete learning situation in the real world, where conditions such as user computer literacy or bandwidth limitations can vary considerably,” according to Barbara Knauff. Over 90% of the peer reviewers use the MERLOT rating.

Unfortunately, to date only a small number of the available materials have been peer reviewed. Trade offs between the volume of materials reviewed and evaluation quality have been inevitable. Nesbit, Belfer, and Vargo point out that, “A costly model that returns highly accurate evaluations is of little use to the individual user if it can only be applied to a small fraction of a collection.”

While the MERLOT team has streamlined the peer review process, each review still takes considerable time and effort.

MERLOT’s peer review process has a second purpose: It serves as a mechanism to recognize contributors who are developing high quality materials. For example, the department heads of Faculty whose work is of the highest quality receive notices. In addition, each Editorial Board selects one peerreviewed resource in its discipline to receive the MERLOT Classics award. The Editor’s Council reviews recipients of the Classics awards and designate those who are to receive MERLOT’s highest honor, the MERLOT Editor’s Choice Award.

Peer review is as critical for corporate knowledge management and learning objects as it is for educational ones. Object consistency, technical viability, and relevance need to be established and maintained if the database is to provide value to the corporation. Mirroring academia, corporations could establish a process whereby a panel of stakeholders would evaluate all submitted learning objects. This would require panel participation by each organizational unit involved in the initiative. Panel members could be learning or knowledge professionals or experts in the line of business.

Nesbit, Belfer, and Vargo recommend a two-cycle review process in which evaluators first independently evaluate the submission and then justify their assessments under the management of a moderator. The choice of a moderator could be problematic since the organizational unit contributing this resource would be losing a significant number of person days per year from their group. In addition, the moderator would need to be credible and acceptable to all participating areas in the corporation.

Challenge Two: Sponsorship and funding

Sustaining MERLOT takes significant funding. In 2002 MERLOT realized total revenues (from grants, partner fees and contributions, and conference proceeds) of $3,391,000 USD. Coincidentally 2002 also saw total expenditures of $3,391,000 USD which, since MERLOT is a non-profit organization, is entirely appropriate.

“A universal issue ... is how to acquire funding and/or revenue to sustain the organization beyond initial project status,” according to Albert Darimont. MERLOT overcame this challenge by structuring the community of practice as a partnership of over twenty institutions of higher education. Partnership takes one of three forms: Institutional Partner, Campus Member, and Sustaining Partners, each of which provides a level of support. Institutional partners contribute $25,000 USD per year along with in-kind contributions to support six to eight faculty members as peer reviewers. The commitment to support peer review amounts to approximately a 25% reduction in workload at the sponsoring institution. Campus members contribute $6,500 USD per year. While this level opens the door for some smaller universities, colleges, and high schools to participate, the cost is still prohibitive for many educational institutions. Lastly, sustaining partners contribute $50,000 USD per year.

In a corporation, sponsorship has two intertwined aspects: executive support and funding. Darimont suggests several possible funding models for corporations. I will discuss three of these. Each of the three models requires support at the executive level. Each of the models could realistically apply to a corporate implementation of a MERLOT-based approach to learning object management. Selection of a funding model would depend on the corporate culture, legal structure, and financial requirements of an organization.

The subsidized model

In the subsidized model, the organization as a whole funds the infrastructure, administration, and initial work to establish schema, content standards, and templates.

Organizational units (departments, functions, etc.) have access to these tools. All content accessed through the Web site links is available to anyone in the corporation. A significant side benefit of this model would be the standardization of approach and content consistency across the organization.

Standardization would ensure transportability, a consistent degree of granularity, and the availability of content for reuse across organizational units. An issue with this model might be sustaining sponsorship and funding commitments. Would the management and on-going administration of the Web site and database remain a corporate priority as markets and organizational leadership change?

The pay-per-use model

In another model, pay-per-use, the organization is still responsible for establishing and administrating the Web site and database but each organizational unit would pay a fee. Considering this model in light of how MERLOT handles funding, it becomes apparent that each organizational unit could be charged a fee for placing objects in the repository or for using objects from the repository, thus allowing the repository to operate as a profit center with revenues being used to sustain its continuous funding. While pay-per-use models require a system or process to levy and collect fees, revenues could cover the cost of such a system. A pay-per-use approach could also serve to increase the value placed on the objects and curtail flooding of the Web site and database with unnecessary or poorly designed objects. A potential issue with pay-per-use is that it makes interacting with the content an operating expense: an overt cost. When budgets are constrained would organization units opt to use internal resources to create learning, even if materials already exist, since employee time is a sunk and static cost?

The member organization fees model

The third model, member organization fees, is the one closest to the MERLOT model. In this model organizational units that want to participate pay an annual fee. This fee allows for cost recovery and provides a stable source of revenue to cover administration and maintenance. Under this model the Web site and database would be self-funding or, depending on the annual fee, operate as a profit center.

Challenge Three: Sustaining participation

Member participation is critical to the success of sites such as MERLOT which, philosophically and technically, are based on a community of practice. Unfortunately, many individual members of MERLOT do not return to the site after their initial visit. Of those who do return only a few access materials and post reviews.

In order for a content management Web site to flourish in a corporation, appropriate and technically viable content must be contributed, reviewed, and used. The MERLOT approach could eliminate one of the barriers to content sharing often found in corporations. But while working with client organizations, the author has observed a high degree of reluctance to share courses, lessons, or even job aids with other units in the same organization. Originating units express a reluctance to lose control of the content and to allow others to use what they have paid for (investment of time and resources) with no benefits to the originator. Using the MERLOT approach each corporate unit would contribute links to objects which remain housed on the originating unit’s Web real estate. Content remains under the control of the originator who also retains the responsibility for content maintenance. Object usage would be tracked and corporate-wide recognition given to units contributing the most widely used, or the highest quality, objects. Depending on the funding structure, organizations that implement pay-per-use would generate funds shareable with originating units. As was seen in the educational context, the quality and quantity of available objects, along with the veracity of the review process, will influence continued participation in organizations over time.

Challenge Four: Administration

California State University handles the administrative and coordination functions required to support MERLOT. It contributes the management, planning, and operation of processes and tools. As membership and participation in MERLOT increases so does the administrative burden. James Rutledge says, “It is recognized that the scope of coordination activities and the requirements for sustaining MERLOT is rapidly increasing, and a new, neutral coordinating organization needs to be established.”

Within a corporation, administration should be closely linked with sponsorship and funding and be centrally managed. As with other enterprise applications such as SAP, a knowledge management or learning content database would likely be controlled and managed by the information technology group responsible and standardized corporate platforms. Conversely a corporation could place the control and management of a learning content Web site and database under the control of the human resources or knowledge management function, with emphasis on the standards and controls required for clear, appropriate, and reusable content creation, management, and dissemination.

Conclusion

The MERLOT approach to learning object contribution, review, management, and dissemination is a viable alternative for many corporations. MERLOT represents an approach to content management that does not require a major capital investment in an enterprise-wide learning content management system. Nor does MERLOT require centralization of learning and knowledge objects under one locus of control. This flexible approach allows for centralized sponsorship, funding, and administration while maintaining the decentralization of content development and maintenance. This flexibility may also decrease resistance to resource sharing between organizational units by allowing each unit to retain control of their intellectual capital. Since MERLOT is, from a technical perspective, a Web-accessed database of links to objects, it is possible to avoid the application integration issues that plague LCMS implementations.

Under this model each organizational unit would access objects to integrate into their learning pieces. They would not need to reach cross-unit agreements on topics, lessons, or courses, only on templates to provide some consistency to those accessing the end-product: the learning. Agreement on, and creation of, instructional design and technical development standards, required schemas, and a peer review process will be a challenge no matter what approach is taken to corporate-wide content management.

References

References related to MERLOT are almost exclusively from educational sources since there appears to be little or no work on-going which explores the application of this approach to corporations. The MERLOT references cited are those closest to the project (for example the MERLOT Web site) and those writers most closely involved in the project or most often cited by those involved in the project.

(Editor’s Note: Because of the age of this article, many (if not most) of the items in this resource list are no longer available online. Those that are still available in late December 2009 may return 404 (Not Available) at later dates. In order to simplify maintenance, we have removed all live links from the list. If specific articles are still needed, we suggest that readers try entering the desired url at The Wayback Machine (http://www.archive.org) to see whether the Internet Archive still has the original.)

Anonymous. (2003). MERLOT: The educator’s Google. BizEd 2 (6), 52-54.

Bratina, Tuiren A., Hayes, Darrin, and Blumsack, Steven L. (2002). Preparing teachers to use learning objects. Faculty and Staff Development. Retrieved October 6, 2004, from http://ts.mivu.org/default.asp?show=article&id=961

Bunter, Donna J. and Ladner, Betty. (2002). MERLOT Multimedia educational resource for learning and online teaching. Retrieved October 3, 2004, from http://www.library.uncc.edu/files/9/merlotdemo_files/frame.htm

Cafolla, R. (2002). Project Merlot: Bringing peer review to Web-based educational resources. Society for Information Technology and Teacher Education International Conference 2002(1), 614-618. Retrieved October 1, 2004, from http://dl.aace.org/10843

Darimont, Albert W. (2002). Environmental scan of pricing models for online content, Report II: Business models for object repositories. Retrieved October 6, 2004, from http://www.edusource.ca/craw/RPT_Pricing_Model201.htm

Degen, Brian. (2001). Capitalizing on the learning object economy. [Retrieved October 1, 2004, from http://www.learningobjectsnetwork.com/

Downes, Stephen. (2000). Learning objects. Retrieved October 2, 2004, from http://www.newstrolls.com/news/dev/downes/column000523_1.htm

Friesen, Norm. (2003). Three objections to learning objects. Retrieved October 1, 2004, from http://phenom.educ.ualberta.ca/~nfriesen/

Hanley, Gerard L. (2001). MERLOT: Tool for CLOE. Retrieved October 6, 2004, from http://lt3.uwaterloo.ca/CLOE/1

Hanley, Gerard. (2003). Online resource: MERLOT: Peer-to-peer pedagogy. Syllabus. Retrieved October 2, 2004, from http://www.syllabus.com/article.asp?id=7780

Hanley, Gerry. (2003). Sharing learning objects: Serving MERLOT to higher education. Retrieved September 20, 2004, from http://tast.merlot.org/

Hobbs, Samantha. (2000). Learning Objects and Construction Components. Educational Technology & Society, 3(2).

Knauff, Barbara. (2001). MERLOT promises efficiency and quality control. Spotlights. Retrieved September 20, 2004, from http://www.dartmouth.edu/~Webteach/spotlights/merlot.html

Malloy, T.E. and Hanley, G.L. (2001). MERLOT: A faculty-focused Web site of educational resources. Behavior Research Methods Instruments & Computers, 33, 274-276.

MERLOT. (2002). MERLOT annual report. Retrieved September 20, 2004, from http://taste.merlot.org/documents/annual_reports/MERLOT-annual_report-0102.pdf

MERLOT. (2003). 2003-2004 Campus partner application for the freshman year participation in MERLOT. Retrieved September 20, 2004, from http://conference.merlot.org/documents/applications/MERLOT-campus_freshman-0306b.pdf

MERLOT. (2004). MERLOT Web site. Retrieved September 20, 2004, from http://www.merlot.org

Monroe, Wanda. (2000). Online instructional materials offered through new partnership. The University Record. Retrieved September 21, 2004, from http://www.umich.edu/~urecord/0001/Sep05_00/11.htm

Nash, Susan Smith. (2004). Learning object production and implementation: UTTelecampus. E-Learning Queen. Retrieved September 21, 2004, from http://elearnqueen.blogspot.com/2004/09/learningobject-production-and.html

Nesbit, John, Belfer, Karen, and Vargo, John. (2002). A convergent participation model for evaluation of learning objects. Canadian Journal of Learning and Technology 28(3). Retrieved October 3, 2004, from http://www.cjlt.ca/content/vol28.3/nesbit_etal.htm

O’Dell, Carla and Grayson, C. Jackson Jr. (1998). If only we knew what we know. New York: The Free Press.

Richards, Griff. (2002). Editorial: The challenges of the learning object paradigm. Canadian Journal of Learning and Technology 28(3). Retrieved October 3, 2004, from http://www.cjlt.ca/content/vol28.3/editorial.html

Roberts, Joni R., and Drost, Carol A. (2003). Multimedia educational resource for learning and online teaching (MERLOT) . College and Research Libraries News 64 (6).

Robson, Robby. (2001). All about learning objects. Retrieved September 16, 2004, from http://www.eduworks.com/LOTT/tutorial/learningobjects.html

Rosenberg, M.J. (2001). e-Learning: Strategies for Delivering Knowledge in the Digital Age. New York: McGraw-Hill.

Rosset, Allison, and Sheldon, Kendra. (2002). How can we use knowledge management? In The ASTD e-learning handbook. New York: McGraw-Hill.

Rutledge, James. (2001). MERLOT Multimedia educational resource for learning and online teaching. Retrieved September 16, 2004, from http://www.spcollege.edu/central/maa/proceedings/2001/rutledge.pdf

Smith, Mark. (2003). Multimedia educational resource for learning and online teaching (MERLOT). Retrieved September 20, 2004, from http://www.bowdoin.edu/~samato/IRA/reviews/issues/jun03/merlot.html

The COHERE Group. (2002). The learning object economy: Implications for developing faculty expertise. Canadian Journal of Learning and Technology 28(3). Retrieved September 20, 2004, from http://www.cjlt.ca/content/vol28.3/cohere.html

Ward, Lynn. (2003). MERLOT overview. IHETS. Retrieved September 20, 2004, from http://old.ihets.org/learntech/merlot/index.html

Weech, Terry L. (2002). MERLOT: Multimedia educational resource for learning and online teaching. Retrieved September 20, 2004, from http://ecommerce.lebw.drexel.edu/eli2002Proceedings/papers/Weech145MERLO.pdf

Werbach, Kevin. (May-June, 2000). Syndication: The emerging model for business in the internet era. Harvard Business Review.

Wiley, D. A. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D.A. Wiley (Ed.), The instructional use of learning objects (pp.1-35). Retrieved January 4, 2004, from http://www.reusability.org/read/chapters/wiley.doc

Young, Jeffrey R. (2000). Merlot project brings peer review to Web materials for teaching. The Chronicle of Higher Education. Retrieved October 4, 2004, from http://chronicle.com/free/2000/06/2000060101u.htm

Comments

Login or subscribe to comment

Related Articles

Marc My Words: Beyond SCORM – A Welcome New Direction

3/8/11

While the LMS and SCORM may not be dead, they do not address the complexities of new learning design strategies and their transcendence of technical standards. Marc reflects on the history of standards and comments on the new ADL Future Learning Experience Project.

Extended Learning Content Management

10/17/11

Instructional design approaches and authoring tools generally embed a certain structure and flow in the resulting eLearning. This creates problems when a designer wants to repurpose content for a different set of learning objectives, or for a different instructional strategy. An extended learning content management system avoids these issues. Read about this concept here!

Viewing Training, Marketing, and Documentation Content Through the Learning Lens

5/16/11

In today’s global and digital information landscape, training, documentation, and marketing practitioners must create content for wired audiences. Already saturated with digital content, customers want information that informs, educates, and solves their problems at the critical moment of need. Here’s how to view your role through the Learning Lens.