Today's chief learning officers need to have access to meaningful information for decision-making purposes. Through a learning analytics dashboard, CLOs can obtain a broad perspective on all the areas they manage.
by Site Staff
February 27, 2007
Today’s chief learning officers need to have access to meaningful information for decision-making purposes. This data should be appropriately organized around a common set of indicators that provide a broad perspective on all areas CLOs manage.
CLOs should have four main quadrants on their learning analytics dashboards: operational, financial, performance and cultural. Achieving success with learning analytics dashboards also requires five primary steps: research learning metrics, identify macro learning constructs, build micro learning indicators, build a process to collect and report, and design technology and templates for support.
Research Learning Metrics
The first step is to learn what others have done, so you don’t reinvent the wheel. As a CLO, an important best practice is to instruct your team members to do their homework before creating highly custom, internal dashboards.
Identify Macro Learning Constructs
A macro learning construct is a small, well-balanced set of broad learning metric classifications that summarize the results of the entire L&D organization. The four macro learning constructs are operational, financial, performance and cultural.
Study the macro learning constructs and ensure they fit with your organization. It is better to have fewer than more, so if some don’t work for you, cut them. Just ensure you maintain balance.
Build Micro Learning Indicators
Micro learning indicators are quantifiable performance measures that are tracked and linked to the macro learning construct. The micro learning indicators are the actual key performance measures for the four learning constructs.
Operational indicators include:
Sources for these data include the learning management system or registration systems. Another great metric here is e-learning use rate. This is the percent of the e-learning library actively used. Research shows it is less than one-third. As such, a lot of waste exists, and opportunities for streamlining are available.
Financial indicators include:
Some measures traditionally are tracked, such as cost per student day and learning and development budget to actual budget. CLOs, however, also should track metrics such as revenue growth, human capital contribution margin (payroll + learning and development expense / revenue), productivity (revenue / number of employees) and learning and development ROI.
Out of these, the most import is productivity, at least to CEOs in how they view learning and development. This is a financial metric that shows how much top-line output (revenue) can be generated per person. The more that number trends positively, the more productive human capital must be. A learning and development dashboard must have this number because improving human capital performance and productivity is the ultimate financial measure of that.
Performance indicators include:
Performance metrics determine how well an organization trains, so even the basic, Level 1 smile sheet data points are included here. A key metric is time-to-job impact. This is a post-training performance metric, and it is gathered on the job. It looks at what percent of people applied the learning and when. Research shows 55 percent of learning is not optimally applied within six weeks. This is wasted training, and removing it is a way to use the metric to rid your organization of scrap and save hundreds of thousands of dollars.
The business results listed here can be tracked in two ways. First, you can do a study to isolate the training impact to the result for a strategic, visible or costly program. Second, just track the trends in business results that are important to the larger organization and/or have strategic learning assisting in their outcome.
Cultural indicators include:
There are traditional metrics such as the percent of personnel eligible for training or hours per employee. But the last three are important too. Management support is collected on manager or participant follow-ups. It looks at how conducive a participant’s environment is before and after training. Internal validation is the number of internal case studies that articulate the success of a major program (try to do one to three per quarter). External validation consists of the articles, awards, speaking events and panels on which learning and development members serve. This shows the internal senior management learning and development is valued by industry peers and also should be valued internally as a strategic partner.
As a CLO, make sure your organization can provide these metrics in a reliable, consistent and timely manner.
Build a Processto Collect and Report
The success factors in this step include:
Begin with data that are credible and easy to gather. Then build a standard template for the macro and micro variables to reduce rework and increase consistency and comparability. The micro indicators might be tracked monthly, but consider reporting them quarterly to senior management, preferably as a trend against actual results and against goals with some type of coding for analytic interpretation.
As a CLO, ask for the process or plan of action to collect and report the data. Ensure they will meet your needs for timely data within the budget parameters you’ve allocated for this type of work.
Design Technology and Templates for Support
The final step is to automate the collection, storage, processing and reporting of what you’ve built. Learning analytics technologies are great tools to build dashboards, input or import results, and track and trend with interpretive color-coding for analysis.
As a CLO, study automation and technology options after you’ve seen the information and used it awhile (at least two quarters). Automate the process when you are confident it is stable and functioning as designed.
You also should review your metrics. Ask yourself, “Am I getting the data I need to manage my business and show value to stakeholders, and am I getting these data in a timely manner?”
When answering this question, remember the operational, financial, technology and cultural metrics discussed. Do these exist in harmony with one another? Are some being missed?
These questions, and their answers, are important. You can only manage what you measure, so measure the right stuff.
Jeffrey Berk is the vice president of products and strategy for KnowledgeAdvisors, a learning analytics technology company that helps organizations measure, communicate and improve the effectiveness of learning and human capital investments through technology and consulting solutions. He can be reached at editor@clomedia.com.