Most learning and development interventions go unmeasured or are measured on dimensions of no interest to the CEO and the CFO.
by Michael Ph.D.
May 29, 2008
Let’s speculate that you own common stock of XYZ Corp. in your 401(k) plan, which creates considerable interest in how successful the company is at producing earnings and improving the price of the stock. But when you read the XYZ Corp. annual report, it mentions information such as how many tons of product have been hauled between warehouses, how many employees have been terminated, how many tons of raw material have been purchased and how many hours of high-speed teleconferencing the organization has done.
When it comes right down to it, you as a stockholder don’t care about the specifics of what the company is doing. Oh, if there is an especially important activity, such as the entry into a new market, then you might like to hear about it. But in general, what you care about are the results, not the activities that create them.
It is no different for top leaders of your company when it comes to learning and development. They really don’t care about the activities of learning and development: the number of courses delivered, the number of hours of instruction, the “butts in seats.” What they want to know is what the outcomes are: what learning and development contributes to the results that matter to the stakeholders of the company.
In the absence of measurements that convincingly show the business impact of learning interventions, the learning and development organization largely is ignored as not being strategic to the company. For better or worse, business impact is the bottom line for learning.
Even in the face of these harsh realities, it remains a fact that most learning and development interventions go unmeasured or are measured on dimensions of no interest to the CEO and the CFO. What gives?
In some cases, it’s like the famous line from the Pogo comic strip: “We have met the enemy and he is us.” The stark reality is that for some, there is a deep fear that if hard, defensible measures are created, they might show that the learning organization’s best efforts are ineffective or that favorable business outcomes simply do not result from the learning intervention. Such confessions are hardly ever part of the public conversation, but the absence of the confession does not mitigate the fact that such fears exist.
In those cases in which management is committed and the resources are allocated to measure and evaluate, several significant barriers exist. What I can share from hundreds of conversations with learning leaders in many dozen Chief Learning Officer magazine Breakfast Clubs and industry conferences is that the barriers boil down to a few important issues.
What is the business impact to be measured? The answer to this depends primarily on two things:
• What is the intent of your learning and development intervention? If it is sales training, for example, the obvious business impact desired is incremental sales — that is, the increase in sales relative to the sales generated in the absence of the learning intervention. This simple example reveals an important feature of measuring, namely, measuring relative to something (e.g., a control group that has received no intervention). The value created by the intervention is the increment, not the absolute.
• What data is available to analyze the intervention and the outcomes? It does little good to identify a business outcome where no data exists. In the absence of hard data, measurement is not possible. Given the current state of measurement and methods, this means that some business-impact measurements are just not possible at this time.
The other question I hear a lot is: How do we prove the outcome measured is the result of the learning intervention and not the result of some other factor(s)?
The answer to this involves control groups and, for the most defensible results, some pretty sophisticated statistical analysis tools.
With the buy-in from top management on which business-impact parameters are important and the use of control groups for comparison and deployment of rigorous statistical methods, the result can be the budget and management support every learning executive seeks. The challenge is to take these general observations and apply them in your organization.