Culturally diverse and geographically far-flung, the new global workforce is leading businesses to search for better, more effective ways to manage growing requirements for employee learning solutions. Clearly, e-learning has become one of the most popular solutions to meet this need. While e-learning enthusiasts extol its lower costs, broader accessibility and personalization potential, e-learning also has experienced slow user adoption and high dropout rates in many organizations. In some cases, users become easily frustrated or unenthusiastic about the material, and if they don’t complete the course, the company might not realize a return on its investment. Usability testing can address some of these shortfalls in the learner experience, as well as provide designers with a set of principles and methods that can be used to design courses that will capture and hold users’ interest from start to finish.
Usability testing has long been a part of the software and product design world. Jakob Nielsen brought the concept of usability to the Web, making Web pages simple to navigate and intuitively organized so that users can easily find the information they’re looking for. While this definition may be considered sufficient in the world of software, the definition of usability in the e-learning world should encompass a few more components than simply good user interface design.
One often-forgotten piece of usability is usefulness. The product not only must be easy to use, but it also should serve a purpose. In the development of e-learning courses, usefulness is measured as part of the needs assessment for the course—a step that often is rushed because of time and budget constraints. These constraints commonly create a tight relationship between the people conducting the needs assessment and those managing the design and development of an e-learning course.
To further refine the definition of usability for e-learning, the term “learnability” also should be added. Donald Norman, known to many as the authority on workable technology, is the originator of learnability. Learnability is defined as the ease and speed with which users can figure out how to use a product. For example, if learnability is high, users can intuitively learn to use a product without training or manuals. In the world of e-learning, the definition of learnability should be expanded to include the ability of users to effectively learn and retain the skills and knowledge. The level of learnability in a course is most often associated with the strengths and weaknesses of the instructional design.
The final component of this expanded definition of usability is the concept of motivation. E-learning that is created with ease-of-use, usefulness and learnability in mind is simple, has high instructional value and is supportive to the learners in their work. However, the elements missing from e-learning, such as an instructor, student interaction and an actual physical environment, can result in a lack of learner motivation.
By weaving in elements of motivation throughout the e-learning design, a more engaging and meaningful experience can be created for the user, resulting in a higher rate of return. The concept can be as simple as understanding what makes a child play with the same toy over and over again. Toy companies must market and advertise to bring kids and parents into the store. But advertising will only get them so far—they must understand what motivates children and adapt their design accordingly in order to create toys that will not end up tucked away in a toy chest. Similarly, companies must promote awareness of their e-learning programs and provide extrinsically motivating rewards, such as incentives and bonuses. However, that will only drive users to the course—what will motivate them to stay and to return for more?
Focusing the user interface and instructional design of training on motivational principles can help achieve built-in motivation that engages learners with the content from beginning to end. (See Figure 1.) Targeted usability testing allows for a better understanding of learners, creating a learning culture made up of intrinsically motivated users who will seek out future educational opportunities on their own.
There are two methods—usability through design and usability through targeted testing—that can help achieve this refined definition of usability. The two approaches should be conducted parallel to each other and in an iterative fashion. (See Figure 2.)
Building usability into the design of e-learning includes:
- Utilizing knowledge gained during the needs assessment to ensure usefulness.
- Employing interface design principles to achieve ease-of-use.
- Using instructional design principles to attain learnability.
However, these principles should not be considered in isolation, but viewed through the lens of motivation principles such as feedback, curiosity, relevance, control, challenge and contextualization to achieve built-in motivation within the e-learning course.
Consider how the motivation principle of feedback might be used in combination with interface and instructional design to achieve different goals. Feedback in interface design, such as visual and audio cues, helps users understand the consequences of their actions, while feedback in instructional design offers guidance and remediation to help reinforce learning points. Combining visual cues such as facial expressions that depict realistic reactions with coaching feedback will create a more authentic learning environment.
By leveraging the motivation principle of curiosity, developers can ensure a course design that creates an environment of discovery, yet is fully comprehensible. Likewise, providing a continuously challenging activity that is just above the user’s current level of competency will keep the learner motivated throughout the course. Using instructional design and user interface principles together to further motivational goals promotes an integrated design process that will better meet the needs of the learner and allow for higher levels of engagement and return.
Just as the design of the course helps ensure its usability, conducting targeted usability tests helps ensure that the e-learning courseware is intrinsically motivating and targeted to the learners who will actually be using it. There are several relatively inexpensive methods that can be utilized alone or in combination to better understand the end users.
Instructional Audit: Testing the effectiveness of the course’s instructional design and learning activities prior to the development of the e-learning course can eliminate the need for costly and time-consuming adjustments. Instructional audits allow for the validation of the instructional design approach with a targeted group of learners in an off-line environment.
In this testing approach, a comparable off-line, classroom model is extracted from the online approach before the testing commences. A minimum of three users should participate in the session, with follow-up several weeks later to assess whether or not they have retained and applied the knowledge and skills from the session. Based on their feedback, recommendations can be developed and iterated into the design.
User Interface Design Testing: User interface design testing is an excellent way to gather preliminary feedback on the user interface of the course. Gathering targeted feedback early in the development stage allows for interface changes to be implemented in a timely and cost-effective manner. The costs associated with this approach are minimal, and the use of online collaboration tools such as Web and phone conferencing, rather than in-person sessions, can help minimize costs. The use of online collaboration tools also allows access to a higher volume of learners across many locations, and can provide additional information that might not otherwise be observed in a testing lab, such as the number of times learners were distracted by instant messages or telephone calls.
Testers in this method should represent a cross-section of the target audience, and storyboards and prototypes should be used to give learners adequate visibility into the navigation and graphic interface. The sessions should be followed by a standardized interview to clarify observations and feedback. Once all of the sessions are held, the feedback gathered can be utilized to develop design recommendations.
Personas: Personas are archetypal learners that represent the needs of larger groups of users in terms of goals and personal characteristics. Developing personas for e-learning can provide the guidance needed to stay focused on the end user and make informed decisions about the functionality and design of the course when actual end users are not available for testing.
Building personas begins with gathering information from sources such as the client’s Web site and interviews to learn about the prototypical users. Developing the persona can be as basic as including demographic information, or as comprehensive as detailing the user’s likes and dislikes. Choose a name and photo to represent this persona, and create a poster to display in a visible place to ensure that whenever decisions need to be made, the user is always “present.”
Workplace Reality Check: The goal of a workplace reality check is to uncover any workplace realities that the client might not know about that could impede the success of the online course. For example, the learner’s workplace environment might have noise distractions, an inconsistent Internet connection, or no access to speakers or headphones to listen to audio.
A member of the courseware design team should plan to spend several hours shadowing a typical user to determine environmental issues that might affect the ability of the average user to consume training. The session should conclude with questions that allow the designer to probe deeper into the observed issues and develop design recommendations.
E-Learning Acceptance Testing: This form of testing is designed to provide a richer picture of how successful an online course will be, in terms of its level of learner acceptance and the level of complexity required. This testing method is especially important when designing online training for inexperienced e-learners.
E-learning acceptance testing should begin with the identification of an online course that is fairly representative of the level of complexity, interactivity, bandwidth and language levels that also will be utilized in the current development project. Users should be observed completing this representative course, either in person or via Web and phone conferencing. Once the user is finished, the observer should interview the learner about the experience with the course, including the ability to consume the level of content and its ease of navigation. The information gathered in this step from first-time users is critical to ensure a successful course for this type of learner. All of the feedback gathered can then be incorporated into the design of the new courseware.
Beyond Usability Testing
The definitive determination of a course’s usability does not stop with this expanded definition, the utilization of design principles and the completion of targeted tests. Usability goes beyond these steps to include post-course assessments, surveys and interviews to gain a deeper understanding of the impact the training has on the learners. The combination of all of these elements will help an organization measure whether or not the e-learning program met the business goals that triggered the need, including the achievement of measurable performance improvement and true return on investment. Post-course feedback will help refine the course for continued administration. As further needs are discovered and new populations of employees are hired, the circle of usability will continue to help develop engaging courses that are usable, useful, learnable and motivating.
Shailesh Shilwant is a lead creative director for Convergys Corp., and has been working in e-learning for the past seven years, providing strategies on making e-learning simultaneously usable and fun. Amy Haggarty is a learning consultant for Convergys with more than six years of experience in the educational technology field. They can be reached at firstname.lastname@example.org.