Chief learning officers face increased pressure from the C-suite to justify expenses and quantify business decisions. It’s time to re-examine how to measure learning and make a change.
Your training classes have a 92 percent fill rate. A whopping 95 percent of your employees have completed one or more formal training programs. You’re thrilled to see these positive metrics — until upper management asks you to define what employees are learning, where they are learning it and how this learning impacts the company’s bottom line.
Chances are you don’t have an answer. Most learning professionals don’t. It’s time for a paradigm shift in the way we approach learning measurement.
Companies are grappling with issues related to learning measurements. According to a survey conducted in 2007 by Expertus and Training Industry Inc., almost 77 percent of respondents said their current measurements fall short in providing meaningful intelligence for business planning. Additionally, only 16 percent say they have all the information they need for business planning and strategy.
Learning leaders aren’t the only professionals who aren’t getting what they need from current learning measurements. According to the survey, most companies still focus on learning-related metrics such as course completions (84 percent), registrations (57 percent) and learner costs (49 percent). However, 69 percent of the audience looking at these metrics includes business-unit executives and managers.
When your audience is focused on cost-savings and revenue growth, telling them about full classes and the number of people trained just isn’t going to cut it. Business-unit managers want to know what their employees are learning, where they are accessing information and how that knowledge impacts the business. Your LMS can’t answer most of these questions. Workplace trends — such as informal learning, the globalization of the corporate workforce and movements toward alternative work environments where staff members telecommute and job share — continue to be hard to measure.
With increased pressure from the C-suite to deliver comprehensive, meaningful intelligence that business-unit managers can use for strategic planning, it’s more important than ever for learning leaders to have “dollars and sense” in mind. Maximize the measurement tools you already have, develop new methods to measure informal learning and deliver the reports in succinct formats.
Maximize the Tools You Have
According to another Expertus survey, the data sources most frequently used by learning professionals for reporting are LMSs (55 percent), HR systems (47 percent) and performance systems (31 percent). Learning managers say these systems often fall short due to hurdles such as data inaccuracy, lack of adequate data, reporting limitations of the technology and limitations in time to gather and analyze the data.
Pointing fingers at the limitations of technology isn’t doing anyone any good. Seventy-six percent of respondents to the survey said they’d like to include performance-related measurements in their reports to the C-suite, and 53 percent of respondents would like to include business-based costs. Both are metrics that would be of interest to business-unit managers and executives.
However, only 43 percent of respondents include such performance measurements — revenues per sales representative or number of resolved calls per representative, for example — in regular reports. A learning leader in line with the needs of the C-suite will look beyond fill-rate percentages and the number of employees who have completed training when compiling measurement reports. They instead will learn to maximize their LMSs, HR systems and performance systems to pull out relevant performance measurements.
Before learning professionals can begin to pull “hidden jewels” from their current systems, they need to know how efficient — or inefficient — they are. Operational efficiency is hardly a sexy topic, especially in corporate training, where delivery always has the spotlight. And because efficiency can be difficult to define and achieve, it’s often overlooked. However, research points to the fact that efficient operations are a key success factor for corporate training organizations. The more efficient the training operations, the more budgets open up for training delivery and development.
It is with this in mind that all learning leaders also must measure their internal efficiency. Where is money being misspent? Where are processes broken or cumbersome? Where are personnel hours being misused? What systems are being underutilized? With a comprehensive efficiency audit, learning leaders have the ability to bolster their measurement capabilities, allowing candid conversations about where resources are being used appropriately and which resources should be reallocated. No one can afford to ignore the value of operational excellence.
Develop New Methods to Measure Informal and Alternative Learning
According to Expertus, 55 percent of learning leaders rely on their LMS for learning measurement. However, in a recent webinar, Josh Bersin revealed that only 10 percent of corporate learning occurs in a traditional learning environment such as with an LMS. On the other hand, 20 percent of learning takes place when learners are seeking information in support of their work, and 70 percent of learning occurs through informal, on-the-job training.
In August 2007, Forrester Research analyst Claire Schooley, with Kyle McNabb and Shelby Semmes, released research indicating that “informal learning continues to gain ground in the corporate training environment as instant messaging (IM), blogs and communities of practice find homes in lines of business. Today’s workers enter the workforce expecting access to the same kind of tools they use in their personal life to help them do work. But there’s a disconnect; most organizations still spend the majority of their training dollars on formal learning even though most employees now learn primarily through informal activities.”
Learning leaders who limit themselves to measurement of formal learning need to think bigger. They are missing out on the vast majority of learning that’s happening in the workplace. The C-suite and business-unit managers want to know how employees are learning — and what they’re learning — through wikis, blogs and other informal channels. But most importantly, they want know how these things are affecting the business.
According to Forrester, a variety of tools now available have the capacity to measure informal learning, such as video on-demand deployment tools, “audio over slides” presentations, on-demand webinars, searchable white papers, wikis, blogs, communities of practices infrastructure and much more.
These tools allow learning leaders to retrieve metrics that measure the information that is being accessed. They also allow training and development professionals to determine who is accessing the information via numbers of downloads, page views and other resources.
Deliver Data to the C-Suite
Training professionals are well-attuned to the education needs of their primary customers — the organization’s learners — but sometimes have difficulty reading the minds and needs of their other customers — the executive team. Tom Kelly, a learning leader with more than 25 years of experience in the education and training industry who has held positions at NetApp, Cisco and Oracle, has spent a good deal of his career figuring out which data is important to management and developing creative methods to mine that data.
“Most of our training metrics are ‘busy-ness metrics,’ not ‘business metrics,’” Kelly explained. “They demonstrate how busy we are, not how much real impact we have on the success of the enterprise that employs us.”
Several years ago, when Kelly faced the reality that his learning organization was measuring and reporting metrics that were not a great value to the C-suite, his team looked to another organizational group as a potential source of data. As an alternative resource for reporting facts and figures, Kelly’s learning organization enlisted the efforts of its own trainees, the company’s sales department, to bulk up reports.
In a joint effort with the sales team, learning was able to demonstrate that, in one training instance — 30 days after most of the targeted sales force completed a training program — the company’s sales numbers shot up. In studies that followed, the two departments demonstrated a similar phenomenon three times across three different product or technology launches. Never again did Kelly’s training staff have to justify an increase in sales-training funding during his tenure as a learning leader.
Other departments can be a valuable source of information as training professionals try to show concrete business impact to the C-suite. Do workflow calendars become more efficient after a time management seminar? Is there a decrease in call-center tickets that require supervisor intervention? Do sales figures climb immediately after product-launch training? These are all examples of measureable, business-focused results that can be pulled from training programs and turned into reportable metrics.
The Unintentional Benefit
By rethinking the way you approach training measurement, you will provide the C-suite with usable information to improve efficiency, drive the bottom line and more effectively run business units. And you’ll also reap an unintentional reward: gaining information you can use to improve the cost-effectiveness and efficiency of your own programming.
According to Expertus research, when making future learning-related decisions, such as course offerings, class scheduling and budgeting, learning leaders most frequently use current training metrics, such as fill rate and percent trained. Yet, as mentioned earlier, 77 percent of these respondents stated these same measurements fell short in providing meaningful intelligence for business planning. So why are learning leaders using the same measurements for business planning as a basis for their departmental decisions?
If you’re using these measurements because it’s “what you’ve always done,” you’re shooting yourself in the foot. Aligning learning measurements to the needs of the C-suite not only will help them make more impactful, knowledgeable business decisions, it will also help you do the same.
A new approach to measurement also can help you add value to your existing learning technology and, at the same time, decrease departmental spending. Once you acknowledge the value of alternative learning sources, such as wikis, blogs and other informal channels, you can capitalize on these new opportunities and use them to your benefit. By integrating informal learning techniques into your training system, you can lower departmental costs such as instructor fees, travel expenses and facility costs, just to name a few.
The Lasting Benefit
Our ultimate goal as learning leaders is to improve the way our company does business. We teach company processes and procedures to new hires, time and personnel management skills to our company’s future leaders, and leadership and interpersonal communication techniques to the executive team. However, in today’s economy, we are under increasing pressure to provide justification for every dollar spent. By taking a realistic look at our learning systems and becoming advocates of change, we create a culture of relentless improvement — one that impacts the quality of our programming and demonstrates our value to the business as a whole.
Gaining the respect of our colleagues, learners and the C-suite: Is there a bigger benefit than that? The result is well worth the effort.Filed under: Learning Delivery, Measurement, Technology