The Learning Lab Development Process: From Idea to Actualization

While we normally consider unbounded thought to be the cornerstone of creativity, the actual implementation of such thoughts and ideas is what creates innovation. Ideas need a place to actualize – a place where they can become more than an electrical impulse jumping the synapses of a mind.

Some of the most innovative ideas incubate in a laboratory-type setting that encourages experimentation and freedom of creative design, leaving room to gain wisdom and insight from mistakes as well as successes. Researchers such as Daniel Pink (author of A Whole New Mind) and David Edwards (author of ArtScience and originator of Le Laboratoire in Paris) actively speak out about the necessity to cultivate creativity, encourage experimentation, and cross disciplinary boundaries to drive innovation. Indeed, organizations spend a lot of time and effort trying to foster such experimentation. This article attempts to highlight one such organization in order to tease out some lessons and best practices that might be applicable to other companies designing learning tools and simulations.

The Wharton School of the University of Pennsylvania’s Al West Jr. Learning Lab routinely relies on experimentation to yield creativity and innovation. Although it is not a lab in the physical sense, the Learning Lab provides an environment for faculty to try new things – an environment where they actively participate in creating their own learning tools. Faculty members formulate ideas for projects that will enhance learning in the classroom. When paired with the technical expertise of the Learning Lab development team, those ideas are realized in the form of Web-based learning exercises and simulations. The experimental nature of the Learning Lab helps to promote educational innovation. My intent is to explain the development process of the Learning Lab, and to share the lessons learned through both the successes and the failures.  

Faculty authors: Freeing the idea 

From its inception in the Fall of 2000 as a project funded by Wharton alumnus Alfred West Jr., the Learning Lab was established with traditional laboratory connotations in mind – a setting in which faculty could freely experiment with ways to enrich and deepen learning in the classroom. While there is no mandate for the use of technology in Learning Lab projects, faculty experiments have trended towards relying on technology as their platform of change. Of course, any project with the goal of transformative learning will entail a large element of risk. In the case of the Learning Lab, the majority of the risk lay in introducing new methodologies or technologies into the classroom. However, such risk also contains the opportunity to create something innovative with a potentially large impact on education. In talking to me about the faculty members who help create Learning Lab applications, Deirdre Woods, CIO of Wharton Computing, stated, “It takes an extraordinary individual to introduce such risk, and to put time and energy into developing a Learning Lab project.”

The challenge begins when a faculty member decides to work with the Learning Lab. The application and support process for Learning Lab projects has three phases: submission of a short letter of interest, selection of projects by a committee, and a review, renewal, or extension phase.

The first step of the process requires faculty to submit a proposal or letter of interest addressing four basic questions:

• What is the goal of the project?
• How will the project enhance learning, either in the classroom or as preparation for students outside the classroom?
• What faculty and/or students will be involved in the development of the project?
• Which course or courses will be the target for deployment of the project?

The Learning Lab Committee reviews the letter of interest. This Committee currently consists of two faculty co-directors, along with three other professors. Together, they evaluate each proposal based on scale, impact, time requirements, and applicability in the classroom. While the committee screens projects on various metrics, the actual guidelines for proposal submissions are intentionally broad to cast as wide a creative net as possible.

As quoted directly from the proposal guidelines:

Proposed projects can make use of technology, but the committee takes a broad view of innovations to learning and teaching that can include innovations that do not rely on technology. Because we are interested in fostering experimentation, we do not place bounds on the types of activities that might prove worthwhile. Future projects might be entirely new creations, or extensions of existing applications, and might or might not rely on a technology platform.

Despite the fact that proposed projects do not have to include technology as a medium, most faculty members submit proposals for Web-based exercises and immersive learning simulations. To date, faculty submissions have resulted in producing over 30 applications, used by seven academic departments in more than 35 Wharton courses, with a total student reach of over 38,000. The departments using Learning Lab applications include Accounting, Business & Public Policy, Finance, Operations and Information Management, Marketing, Management, and Statistics. The applications themselves span a diverse range of categories including auctions, real-time trading, complex modeling, prisoner’s dilemma simulations, “tragedy of the commons” scenarios (see the References at the end of this article), single player setups, synchronous multiplayer competitions, turn-based applications, online tools, and courseware. Beyond Wharton classrooms, the Learning Lab’s commercial product, OTIS (Online Trading and Investment Simulator), reaches approximately 2,000 students per semester in more than 80 colleges and universities in the United States, Canada, Colombia, New Zealand, and Hong Kong. 

When asked, during a telephone interview, whether the proposal process of the Learning Lab granted the freedom to exercise creativity, faculty author Professor Maurice Schweitzer responded, “Yes, and I had an enormous amount of control over the process and implementation.” (Schweitzer is Associate Professor of Operations and Information Management, and the author of the Oil Pricing EQuilibrium simulation, or OPEQ.) 

Professor Andrew Abel, co-director of the Learning Lab Committee, states that faculty creativity was an important consideration, even in the nascent years of the Learning Lab:

We had faculty creativity in mind from the beginning. Actually, the creativity comes at two levels. The initial idea is almost always faculty-generated. Most proposals submitted by faculty are not completely spelled out, so that second level of creativity is provided by the Learning Lab, which works with the faculty initiator to shape the idea. Even in those cases in which the faculty submits a very detailed proposal, the Learning Lab adds creativity in designing the interface. (Professor Abel is faculty author of MacroSim, and Ronald A. Rosenfeld Professor of Finance and Economics.) 

To aid the decision process during selection, committee members may ask faculty to provide more detailed information. If a faculty member’s proposal is not selected, he or she may rework the proposal and submit it again during the next proposal cycle. Faculty may also collaborate; in fact, many of the applications have multiple faculty contributors. The Learning Lab accepts proposals every April, and the committee finalizes selections shortly thereafter to allow development time for projects expected to launch in the fall semester. Once a proposal is chosen, the faculty member shifts from the role of idea generator to faculty author. Becoming a faculty author means working closely with Learning Lab developers to convert the idea into a working application.

Learning Lab adoption: Actualizing the idea with creativity and collaboration

As soon as the Learning Lab committee approves a list of proposals, Learning Lab Director Alec Lamon distributes the projects amongst the development team with an eye towards skill sets, availability, interest, and towards familiarity with either the content or the faculty author, or with both. On occasion, the Learning Lab outsources projects due to resource or time constraints. 

The development team can choose from an array of useful tools such as Adobe ColdFusion, Adobe Flex, Microsoft SQL Server, Adobe Dreamweaver, and Adobe Eclipse to name a few of the most frequently used. A set mandate does not exist on the list of applicable tools; in fact, developers continuously suggest new tools that will help increase the efficiency of development, or serve to better demonstrate the desired learning principles. However, the development team has learned the hard way that technology choices must always adhere to the learning goals of the project. According to Alec, “It is crucial that we cast a very critical eye on new tools and methods before adopting them – we’ve gotten stuck in the weeds before by choosing a hot technology over an appropriate one.”  

The Learning Lab developers and faculty authors work together to transform proposal ideas into concrete applications. David Edwards (see References) points out:

Just as a scientific lab is only as successful as the scientists who work within it, an artscience lab counts on the innovation inherent in the minds of the collaborating artists and scientists. It is a network, like the Web, which occasionally correlates the creative efforts of large numbers of individuals.

Using this metaphor, the success of the Learning Lab arises from the innovation inherent in the minds of the collaborating developers and faculty authors.

I enjoy digging into the models that these professors develop. They’re world-class thinkers, and leaders in their fields, so their ideas are not necessarily easy to implement, but it is exciting to be working on a Web application that does something unique (Cadence Anderson, Senior Programmer Analyst, Learning Lab, in an e-mail message to the author, September 2, 2008).

While the proposal process provides plenty of space for faculty creativity, the application development process grants creative freedom to the developer – a freedom the Learning Lab staff cherishes. Cadence Anderson commented, “I like the fact that creativity is not stifled and innovation is encouraged in this environment. It’s nice to know if you want to try a different approach or a new technology, you have the space to do that.”

Developers choose interface designs, color schemes, and multimedia that meet faculty proposal requirements, yet still reflect their own visual and creative abilities.

At the Learning Lab, we are already a group of very creative people. Our creativity manifests itself in our ability to design new Web interfaces that are both user friendly and aesthetically appealing. We are innovative in the way we manage input from both professors and students on how to improve existing applications (Erin Wyher, Senior IT Project Leader, Learning Lab, in an e-mail message to the author, September 2, 2008).

Because faculty members are the driving creative forces behind Learning Lab applications, faculty expectations and requests are at the forefront of the developer’s mind at all times. However, the developers do devote attention to learning theories and learning preferences as well, since this research can also play an integral role in application success. For example, Marc Prensky, propagator of the term “digital natives” and author of the book Digital Game-Based Learning, cites what he sees as ten main cognitive style changes that he observed through research on the current generation of students. Regardless of whether modern students physically have different brains, or whether their preferences have simply been shaped by the abundance of educational tools available, these style changes are important in application development. According to Prensky, the modern student prefers:

• “Twitch speed” vs. conventional speed
• Parallel processing vs. linear processing
• Graphics first vs. text first
• Random access vs. step-by-step
• Connected vs. standalone
• Active vs. passive
• Play vs. work
• Payoff vs. patience
• Fantasy vs. reality
• Technology-as-friend vs. technology-as-foe 

Given all of these considerations, and depending on when the proposal enters the Learning Lab workflow, development requires on average between two and six months of effort from one full-time developer. The development process includes prototyping, testing, and deployment. During the final phase of testing, the developer uses a beta version of the application in class with the professor. After shaking out the final wrinkles during the beta process, Learning Lab deploys the application to production, and developers take suggestions from professors for changes on a case-by-case basis. During the development cycle, each originating faculty member spends approximately five to ten hours collaborating with the development team.

The nature of idea experimentation: Testing the idea

Professors repeatedly use many of the Learning Lab applications each semester. With the permission of the faculty author, an application may be incorporated into the curriculum of other courses or other sections of the same course. Sometimes student groups ask to use a faculty member’s simulation for extracurricular learning. For a summary of some of the most frequently used Learning Lab applications, see Table 1.

Many of the applications enjoy use every semester. However, some find only one use before they return to the Learning Lab memory bank, and occasionally an application never lives past development. Whatever the shelf life, each application reveals lessons related to technology and learning in its own way. By establishing the Learning Lab as a traditional lab environment of experimentation, authors expect to learn from mistakes as well as to gain insight from the more successful applications.

Dr. David Edwards explains the experimentation process at Le Laboratoire as follows:

We perform an “experiment.” We learn. We have started to discover. We do a second experiment. We are in the “translation” phase of idea development. Perhaps this second experiment carries us in a completely different direction. Each experiment or experience moves us along, until, eventually, we meet some barrier. It would be normal for us to stop here, drop the idea, and take up another. But for some reason, perhaps related to the passion we feel for our idea […] we make a decision to step over this cultural or institutional barrier; we jump into different realms of idea exploration. (See References.)

The deployment of Learning Lab applications follows a similar course. Someone introduces an application. Learning occurs. Another application introduced in a different course reveals completely different insights. Occasionally the process meets with barriers, which we will discuss later, but the proposal cycle continues to occur annually, and people always submit new projects. From 2005 to 2006, the number of proposals increased by 50%; each year the Learning Lab receives approximately five to nine proposals for new applications, or enhancements to existing applications.

For David Edwards, “process matters more than results, and results are never bad,” as stated in his laboratory principles. The same laboratory principles hold true for the Learning Lab.

Process matters more than results: The development process rewards both the Learning Lab and the faculty author. The developers exercise their creative and technical expertise, while the faculty author watches his or her idea expand from a concept into a reality. The Learning Lab members gain knowledge of applicable business and teaching models, while the faculty authors view technologies and techniques in ways that were perhaps previously unknown to them. 

The opportunity to work with faculty application authors has been a bonus, since it provides insight into how professors view the teaching and learning process. We collaborate with some extremely intelligent people, and it’s personally rewarding to learn from them (Rebecca Sweger, Sr. IT Project Leader, Learning Lab, in an e-mail to the author, September 3, 2008).

Results never are bad: Learning Lab applications have open-ended outcomes by design, but outcomes also vary because of the differences between professorial teaching methods and student characteristics. Most application runs resemble experiments, and many times the students are research participant equivalents. Each participant enters the process with his or her own preferences, expectations, and experience levels in regards to learning technologies. As a result, simulations sometimes yield unanticipated results due to the random variation amongst students.

Professors design the simulations to have open-ended outcomes from the start, in order to facilitate interesting discussion in the class. Most simulations require a “debrief” session in which the faculty member reveals the pedagogical motivations behind the simulation to the students. “Most of our applications do not presume to teach,” says Alec Lamon, “rather they form a shared experience that provides the basis for an extremely rich and engaging in-class discussion.” Similar to other experiments that involve minimal deception techniques, the object of the simulation does not always reflect the overall teaching objective, which the faculty member reveals after the experiment. Debrief sessions differ depending on which faculty member teaches the course. A debrief is personal, and provides an opportunity for professors to use application data in a creative and sometimes quite theatrical way.

For example, in the Learning Lab’s Oil Pricing EQuilibrium simulation (OPEQ), students assume the role of an oil-producing country, and set production levels in a competitive environment with little or no knowledge of their competitor's intentions. At face value, the simulation is about maximizing profits and setting prices by manipulating production levels. At the heart of OPEQ, however, important lessons emerge regarding negotiation skills and the behavior of competitors in a closed market. The debrief sessions often lead to heated discussions, where students adamantly defend their actions or sharply question the actions of a competitor. 

When asked during his interview if OPEQ extended learning beyond the classroom, and helped to convey a pedagogical point that could have been difficult to explain without technology, faculty author Maurice Schweitzer responded:

OPEQ allows you to change parameters for different students [i.e. Undergraduate vs. MBA vs. alumni]. It is an extremely flexible system, which marries the ideas of competition and negotiation with technology, allowing people to become emotionally involved through the technology. We used to implement the OPEQ scenario using paper and pencil, but it was slow and imperfect – people made mistakes, long response times led to disengagement, and we couldn’t have as many rounds. The technology made the scenario much more powerful.

Validation of a simulation occurs via faculty feedback (acceptance, adoption, and expansion), external adoption (outside of the original course, or, potentially, outside of the Wharton domain), and through data collection and assessment. In the past, the Learning Lab distributed surveys to students measuring overall satisfaction and utility of the applications. See Figure 1 for pie charts of past survey results.

 86% of students said that learning was enhanced by the computer/Web-based tools, 83% of students said that the computer/Web-based tools increased attention and engagement, and 78% of students were satisfied with their experience – only 5% expressed dissatisfaction (the rest were neutral). – Learning Lab Survey Data.

Comments

Login or subscribe to comment