At the conclusion of an ROI Methodology workshop we hosted in Montreal, one of the participants asked us, “Are you teaching us to develop ROI thinking?”
“Yes,” we answered. “You must understand the results of the work you do.”
That being said, it’s important to remember and respect the value chain for the success of any project or program you implement. The figure below shows the five levels of outcomes (plus level 0, which is input).
An estimated 95 percent of the learning and development functions worldwide accept this value chain of outcomes, and 90 percent of evaluation systems are based on this logic model.
With a logic model, one level is necessary for the next level. You no doubt have experienced this many times. You know that if people don’t learn, they obviously can’t use what they have learned, and the process breaks down. If they don’t apply what they’ve learned, there’s no impact. If there is no impact, it’s a worst-case scenario — a negative 100 percent ROI.
The relationship between levels 1 and 2
The good thing about levels 1 and 2, or reaction and learning, respectively, is there is often interaction between the two. Learning influences the reaction, and reaction creates a desire to learn more. When feedback is captured after participants have learned, the feedback includes reaction. If a participant doesn’t see value in what they are doing, and it’s not important to their success, then they won’t use it and the program fails.
Level 3 is important
Your challenge is to push the measurement to level 3, or application, always thinking through what the participants will do or use. Many times, it’s obvious: You want them to use what they’ve learned. Sometimes you want to stop the evaluation at level 3, with participants using the content. But that’s not really what you need because they need to be applying what they’ve learned toward a purpose. That’s the consequence, the impact.
Application without impact is just being busy.
Level 4 is critical
Impact is the favorite outcome measure among top executives. Because of this, you need to start with the end in mind, a specific impact measure. This upfront connection to a business measure is rarely achieved.
Many fall into this trap with soft skills. For example, if participants are in a communications workshop, and they are communicating, that’s all you need, right? Not really. If the communications do not have a consequence, there is no impact, and you haven’t made a difference. The definition of making a difference is to have application and impact. Will the communications cause them to be more efficient (time savings), reduce errors (quality) or help to get more done (productivity)?
Even if the consequence is an intangible measure that you don’t convert to money, it’s still important. When you have an actual impact, you must sort out the effects of the program from other influences to be credible.
ROI — is it worth it?
Finally, there is the ROI. Some individuals say it’s inappropriate and unnecessary. Yet, those same people use ROI thinking when they make a purchase. Everyone buys products or services, and constantly asks themselves, “Is it worth it?” You may not have a calculation, but you conduct a mental benefit-cost analysis. It costs this amount, and this is the value you will receive. This is an important question to ask, and it guides major funding decisions in organizations as well.
In L&D work, it’s all noncapital investments. The executives know a process to show the value of capital investments, but capital spending is not the major expense anymore. Now the major expenses are noncapital, such as people, IT and marketing. The concept of ROI for noncapital investments has grown significantly in the past two decades, often driven by top executives and chief financial officers. Calculating the ROI for a major L&D program is appropriate and necessary when setting priorities and showing the efficient use of funds.
ROI thinking
ROI shouldn’t be something to fear but should be embraced and used properly. Use it for those programs that need to make a difference and for which executives need to see the outcomes compared with the cost. Everyone needs to know if the program or project was worth it.
More important, think about the chain of value on a regular basis for everything you do — develop a habit of ROI thinking.
