Are you a learning development professional who struggles with increasing your customer’s satisfaction for your online training? Does your organization outsource course development to a variety of vendors, only to receive mixed results in terms of quality and cost? Is your management putting pressure on you to deliver courseware faster and cheaper?
If so, I'd like to show you how Microsoft’s Sales Marketing Services Group Readiness (SMSGR) organization has developed solutions for these issues.
SMSGR is a training organization in Microsoft comprised of approximately 150 individuals located in regions all across the world. Our headquarters are in Redmond, Washington. Within Microsoft, SMSGR is responsible for improving performance of the “Field” employees, approximately 32,000 sales, services, and marketing professionals throughout the world. SMSG Readiness offers dynamic, innovative, and high-quality learning opportunities.
Our learning programs assist the Field employees in delivering on their commitments, being more productive, driving results, and growing their careers. Through strong partnerships with learning and readiness teams across Microsoft, SMSGR drives a learning platform that consolidates all required and optional training from across the company, and makes it available to the Field. Individuals can achieve learning objectives through a variety of methods, including attendance at SMSGR-powered events, participation in instructor-led or online classes, and completion of self-paced instruction. In addition, we constantly update Field readiness resources with new and timely content, such as solution selling strategies, discussion guides, and Live Meeting recordings for topics relevant to specific disciplines.
To complement Microsoft Field’s Career Development Plans, SMSGR provides valuable tools, such as Role Guide™ and Career Compass™, that help individuals prioritize learning opportunities and create an individually tailored, holistic training roadmap. This, in turn, allows employees to deepen skills in their current disciplines, and develop new competencies that pave the way toward professional growth and career advancement. Furthermore, Field employees can self-assess whether a learning opportunity is worth their time, or, conversely, test out of courses that cover content that they already know.
Business problem
This year we enhanced our delivery of learning programs to the Field by addressing several customer issues, and developing a simple and easy-to-use interface for our online courses. In order to improve our services, we conducted interviews with our customers, collected feedback in business meetings, and made an analysis of our help-desk service requests. We discovered several business issues that we needed to address:
- Quality and consistency of our courses
- The cost efficacy of our outsourced vendor model
- Customer experience
- Customer satisfaction
- The maintenance of our learning programs
Course quality: Customer experience and satisfaction
At SMSGR, we outsource the development of our courses to more than 50 vendors. Consequently, we were delivering courses that varied significantly in design, navigation, and adherence to accessibility best practices, compatibility with our learning management systems, assessment strategies, and aesthetics. Each vendor designed their courses in a unique style. Our customers complained that for each on-line course they had to learn how to take the training before their learning could begin. We were also experiencing inconsistent adherence to Microsoft style guidelines and accessibility best practices (i.e. compatibility with the Section 508 of the Americans with Disabilities Act [ADA]). Sometimes the results were unacceptable. Customers were also concerned with the length of courses, and how much of the material was directly applicable to their role. For example, one online course was one hour in duration, but only 15 minutes was applicable to the customer's specific job.
Our SMSGR professional curriculum managers are dedicated to improving our courses by implementing best practices from lessons learned in previous courses. However, we noticed that adoption of best practices was on a course-by-course basis, and implementation of innovations had no sharing across the curriculum. While these complaints applied primarily to our online courses, we received similar complaints about the inconsistent approach to the design and curriculum of instructor-led courses.
Cost efficacy
Managing the overall cost of building high quality and engaging training experiences is essential to maintaining our ability to maximize SMSGR productivity and output. With more than 50 vendors, we struggled with the variance in quality and cost of their deliverables. Engaging vendors became challenging and risky, as we relied on their design and development skills to meet our customers’ needs and our deadlines. It was clear to us that we needed a means to identify vendors that have high quality products and services, that map to SMSGR’s requirements, and that have agreed to align with SMSGR’s business model. We needed to achieve long-term cost efficiency by aligning the vendor to SMSGR standard rates for delivery and development. We also needed to focus more on sound instructional strategies, and less on compatibility with our platforms, accessibility, and other aspects that would come with a standardized approach to delivering learning.
Localization
SMSGR delivers courses to a worldwide audience. While English is the primary language for course delivery at Microsoft, it is frequently necessary to offer courses a variety of languages. When we needed to localize courses, whether they were online or instructor-led, the courses required re-programming or repackaging of the instructor-led course materials. There was no single way to cost-effectively translate courses that did not require the course to be rebuilt.
Maintenance of learning programs
We have an e-mail alias for all of our learning programs. This serves as our help desk for our learners. We were receiving hundreds of service requests per month for support on course registration, access to materials, video speed to download, assistance on the functionality of online courses, and course completion status. To resolve most of these issues, the help desk team would have to consult with the SMSGR curriculum manager, who would have to contact the vendor that developed the course. Most of the time, the vendor had to do research and work with our Learning Management System (LMS) team to determine what was causing the issue before they could resolve it. A common occurrence was that the course was incompatible with the LMS, and not relaying or recording course completion status properly. Uncovering the root of customer issues was a long process, and customers were obviously dissatisfied with the response time. It was also a frustrating process for our internal staff.
Our solution
After carefully listening to our customers, we determined one solution that would alleviate most, if not all, of these issues for us. Our approach was to provide a consistent and high quality SMSGR courseware format, with the same “look and feel” for courseware across multiple modalities, to provide a consistent user experience. This solution encompassed setting standards for a number of things:
- Course knowledge levels
- Instructional design
- Assessment strategy
- Branding
- Localization
- Vendor engagement
- Courseware materials
- Courseware programming
- Backend functionality of the online courses
|
Business Problem |
Issue |
Our Solution |
|
Course Quality: Customer Experience and Satisfaction |
Customers have to learn the interface before they can begin to learn the content |
Common user interface and navigation components |
|
Adherence to Microsoft styles and accessibility standards |
Embed style and accessibility components in a standard courseware format |
|
|
Applicable content to customer role |
Standard course template with individualized tracks for roles and test-out feature |
|
|
Best practices implemented as one-offs |
Standard instructional design and assessment strategy |
|
|
Cost Efficacy and Localization |
Disparate vendor quality and costs |
Standard templates for courseware, and require vendor certification |
|
High localization costs and work effort |
Embed localization standards in courseware templates |
|
|
Maintenance of Learning Programs |
Large volume of service requests with LMS compatibility issues |
Embed standards for courseware programming and backend functionality with SMSGR LMS through consistent exit button, embedded APIs, and course status recorded frequently to LMS |
Table 1 outlines the business issues, and how we solved them in the design of our solution. To improve learner satisfaction in the online course experience, we developed a common user interface that employs a standard instructional design with consistent navigation elements. Our course template can contain multiple learning tracks based on role or industry (see Figure 1). The implementation of these elements provides the learner with a predictive experience all the time, no matter who develops the course.
Figure 1 Multiple tracks within the same course
There were two ways we were able to address the issues of disparate vendor quality and costs. First, by producing a standard template and requiring vendors to use it, we were able to control the quality of the online course in terms of design and navigation. Secondly, with a programming standard we can minimize the time and resources required to translate a course into another language, thus reducing our costs. We also implemented this approach to improve our ability to maintain our learning solutions.
Implementation
We formed a SMSGR virtual team to guide the development and implementation of our solution. Our virtual team consisted of several different curriculum managers, members from our platform team, and a few vendors to support our development and testing of the online template. As depicted in Figure 2, we identified and conducted several simultaneous work streams for this project. The courseware design component of our solution focused on setting standards for course knowledge levels, instructional design, assessment strategy, and navigation design. Led by our learning strategists, we developed specifications and definitions for various course levels including fundamental, intermediate, advanced, and expert knowledge levels. Each level was defined by description, learning objectives, assessment strategy, and content design guidelines. We held meetings to introduce these to our curriculum managers, made revisions, and received executive approval for implementation.
In order to address courseware development standards, we concentrated on branding, localization, courseware programming, and backend functionality of the online courses. For these efforts, we leveraged our vendors to help develop the template to our specifications. We began with a first version of the online template that we released in July 2007, and then continued to make improvements on the template based on feedback from our users’ tests with our Field customers. We released a series of software patches as we incrementally improved the template. Four months later, we released our second version with additional navigation and user interface features.
I must admit we were literally building the plane at the same time we were trying to fly it. I am sure you can identify with this if you have been working in training for any length of time. The key to our success was dedicated executive support, a terrific project manager, very talented team members, and our ability to be very agile.
Our third major area of concentration was internal communication and education. This work included creating awareness among our stakeholders, training on the template, providing regular communications on progress, and promoting our template to other Microsoft readiness organizations. We held workshops, live meetings, and briefings to educate our internal SMSGR curriculum managers on the instructional design requirements, assessment standards, and the new online template.
To date, we have trained approximately 230 people on the standard online template. Because of our promotional efforts across Microsoft, we have six other readiness organizations adopting our online template. Course developers outside SMSGR have produced more than 20 courses utilizing our template.
The fourth major effort, and the most time-consuming task, was educating our vendor communities. It was critical to our success that we trained our vendor pool on the new course standards. First, we developed a vendor portal, a process for communicating to our vendors, and software development kits for the template. We notified vendors that in order to continue to do business with us they would need to prove they could master the online template.
We then set up a certification process that required vendors to produce a sample course in our template, and submit it for our approval. In turn, we provided software development kits, documented instructions, and e-mail support for questions. We held regular office hours to support the certification process. Our virtual team would test the course and review the programming to ensure it met our guidelines. We have trained and certified more than 25 vendors in our SMSGR online course template. Our vendor management lead continues to hold weekly live meetings and office hours for vendors learning and developing in our template. Within SMSGR, we have released more than 100 courses in our online template.
In the six months between July 2007 and February 2008, we developed, tested, and released a simple and easy-to-use SMSGR-branded online course template programmed in HTML with these integrated features:
- A Main Menu that provides a “due-North” all-up view of the course modules and topics (see Figure 3)
Figure 3 The SMSGR template Main Menu
- A scalable template that can accommodate 1 to 500+ pages in 1 to 12 modules
- A progress window that presents real-time progress information (see Figure 4)
Figure 4 Progress tracking for the learner
- Video encoding and playback standards
- Variable video playback speeds: .75x, 1x, 1.25 to all courses
- Test-out feature that provides opportunity to demonstrate proficiency without clicking through a course
- “Tracks” give user ability to select focus areas
- Captioning and transcripts, available navigation with keyboard, intelligent use of audio, and adherence to Section 508 of the ADA best practices
- Localization and regionalization designed into the template; template is pre-localized in Japanese
- Capability to take away the complete course content as a OneNote or Microsoft Word document, so learners could leverage the content at any time without having to go back to the online course
Finally, we were able to resolve company course-maintenance issues by embedding enterprise-ready application programming interface (API) requests into the course template. By improving the writing of user status to LMS, so that it happens frequently and network interruptions don’t affect the learning system’s ability to track the user’s status, we were able to reduce help-desk support. Another simple fix that resolved issues of user tracking and completion status was making consistent navigation, with the Exit Course button in the same place every time.
Benefits
Through the standardization project in SMSGR, we have addressed and resolved all of the business issues we targeted. According to SMSGR General Manager, Dustin Grosse, "Our customers are providing positive feedback on their experiences on our platform, and the net satisfaction scores and readiness metrics are continuing to increase as a result of the work that is being done in this area."
By providing a consistent user online experience, our customers can get down to the business of learning and not have to waste time with learning the navigation. Through features such as the progress window, our customers know how much time they need to invest as they complete sections of the course.
Our consistent approach has drastically improved our customers’ satisfaction in checking their knowledge of the course material. By providing multiple tracks of the course, differentiated by customer role or market focus, and a consistent module, listing calls of action for customers, we have given time back to the customer and we deliver just enough training for each role. Finally, the test-out feature added to our courses permits our customers to determine whether they need to take the course.
We have realized many benefits with our outsourced vendor model. By requiring vendors to use, and be certified, on the template, we have improved the overall quality of our online courses and reduced the time it takes to develop them. As such, we are gaining more predictability on the costs required to produce an online learning solution. Our online template is now maximized for localization such that we can quickly turnkey the translation of course content, and quickly provide a localized course. This helped us compress localization cycle time and reduce costs.
The standardized courseware template has enabled us to reduce the risk when engaging so many vendors. Jordan Montgomery, SMSGR Manager of Analytics and Vendor Programs, explains, “The outcome of implementing standards is truly a win-win scenario for both the vendors we work with and our organization. The decision to implement standards was well received by our vendors. They quickly recognize the benefits of a consistent approach that provides the opportunity for a predictable and efficient engagement with our organization. They also appreciate that the standards enable the vendor to direct the majority of their time and energy to building strong content that is effective in achieving the targeted learning objectives, which is where the vendor is able to add the most value. Standards-certified vendors are able to apply their deeper understanding of our learning strategies, and their demonstrated proficiency with our tools and processes, to streamline the end-to-end solution design, develop, and deploy process, and hit the target on the first attempt, reducing the time and cost required to build and deploy the learning solution. So the resulting benefit for SMSGR is the ability to engage standards-certified vendors with a higher degree of confidence that the vendor can build and deploy learning solutions in a manner that meets our commitment to achieving significant business results through effective learning programs, while also deploying those programs on time and within budget.”
As mentioned earlier, we were able to reduce the amount of resources and effort required to provide user support for our online courseware. SMSGR Director of our Learning Platform, Jim Federico, summarizes the impact our standards project has had on the maintenance of our learning solutions, ”By listening to our customers, and leveraging off-the-shelf Microsoft technology, we have been able to ratchet up our impact while dramatically improving our ability to consistently deliver high quality and engaging learning experiences. It is not often that an idea so simple can have this significant an impact.”
Future directions
As I mentioned at the beginning of this article, SMSGR keenly focuses on improving the performance of worldwide sales, services, and marketing professionals by offering dynamic, high quality, and innovative learning opportunities. The implementation of the SMSGR standard template has created a venue for a consistent, predictable customer experience, and that has resulted in improved levels of learner satisfaction. It is important to note that standards do not always translate to removal of innovation and creativity. We have been very careful to leave ample room for creativity in our template. Our virtual team was committed to balancing consistency and creativity in the design and implementation of our standard template. Vendors can integrate virtually any design and interactivity they desire inside of the template.
The SMSGR Standards project has taken on a new life of its own. Through this project we have identified other improvements that will make our services even more valuable to our customers. We are dedicated to providing our customers with dynamic and innovative learning opportunities. As we embrace our new standards approach, we are working to apply it to multiple modalities. Our team has already developed, tested, and released an instructor-led template for facilitator’s guide, student workbook, and PowerPoint presentations. In the spirit of accommodating different media elements, we developed a one-page SMSGR online template to accommodate the publishing of self-contained instructional content, such as standalone videos, recorded live meetings, and job aids.
As we enter the next phase of this project, we will look to add sizzle to our training by leveraging Microsoft® Silverlight™ technology. We are adding collaborative capabilities within our courseware that enable individuals to collaborate peer-to-peer and within the context of a learning community. The SMSG employees are particularly excited about an innovation we are building that enables them to take any course produced within the template offline. The offline scenario enables learners to synchronize their progress and completion status the next time they connect to the corporate network. Going forward, we plan to leverage advances in Windows Mobile that will enable our occasionally connected workforce to be productive, even during downtime. Additionally, we will expand the test-out feature to allow for testing out on a module-by-module basis within a course, allowing for a higher level of adaptive learning.
Tom Moretti, SMSGR General Manager of Planning, Design, and Development, describes our future plans and vision by saying, “SMSGR has made great strides in raising the quality bar of our online courses through the use of the standardized course template. We’ll continue to add new innovations to the template in response to our Field customer feedback and needs. We envision an environment in the near future where our Field personnel in sales, marketing, support, and services can effortless incorporate learning into the very fabric of their daily work tasks. We can enable them to leverage high-quality learning that is relevant to the individual and the task, while being available at the right time, the right place, and on the right device. This is how learning should be in any company. We live in a dynamic world, and our learning should be part of that dynamic environment.”
