Defending Your Training Budget: ROI and Other Important Metrics

The old saw, “When revenue is down, training gets cut,” is for many CLOs, and their direct reports, a painful reality. So chances are, you are either anticipating cuts or already know their extent as reflected in your final numbers.

The extent of your pain – great or mild – is probably related to how well you have or think that you have defended your budgets. This article is about helping you find the hidden opportunities in the form of key metrics and strategies that will make your future budgetary defenses as strong as possible.

Uncovering Metrics
To start uncovering these metrics, many learning executives use elements of Donald Kirkpatrick’s model of evaluation. You could think of this model as a continuum of evaluation “levels” arranged from “soft data” to “hard data” or from “Opinion” to “Observed Results.”

• Level I: Reaction. The data at this level comes primarily from “self-report” surveys or testimonials from users and addresses the question, “Do they like it?” This is usually soft data and is an inappropriate metric to use for budgetary defense. It plays well in marketing materials, but not for the CFO.
• Level II: Learning. This data is from pre- and post-assessments and shows the impact of training over time, addressing the question, “Do they get it?” This is good support data for your training program, but because it is general in nature, it is probably not appropriate as a budgetary defense.
• Level III: Behavior. Is there observed positive job performance, and is it due to the training? This addresses the question, “Can I observe them doing it?” This is harder data as it relates to observed job performance and could be an important factor in defending your budget, but it is not a metric. There may be some mild interest from the CFO on this one.
• Level IV: Results. What is the impact of training on the organization? This data answers the questions, “Do they use it, and does it make a difference?” This is the most important level, but it is also the most difficult to quantify. Therefore, like Level III, it is not a metric. Some real interest here by the CFO, but expect the follow-up question, “Can you give me some hard data on that?”

Using Kirkpatrick’s model as launching point to get us to a metric that you can use to defend your training budget, we have to acknowledge two principles within the model. First, you must be able to see the behavior in order to measure it (Level III). Second, you must be able to quantify in metrics important to the CFO. This is where Level IV falls short. While this level asks the question, “What is the impact of training on the organization?” it does not provide the answer in the form of a meaningful metric. This is an ROI question.

Human Capital
The term goes back to 1964 and was coined by economists T.W. Schultz and Gary Becker. Originally, Becker and Schultz defined it as the consequence of on-the-job training and health on earnings and economic productivity. The term became popularized with the publication of Alvin Toffler’s book “The Third Wave.” While the Industrial Revolution (the second wave) valued machine and muscle, the Information Age (the third wave) valued information and the people who held it. Pundits quickly made the workers of the third wave and “human capital” synonymous.

The problem with both the Becker/Schultz definition of human capital and the pundits’ definitions is that they did not produce a hard metric that measured the value of the skills, attitudes and behaviors that people possessed. What we were left with was a valuable Information Age concept, but without the financial rigor of the Industrial Age.

Fixed Capital and ROI
Fixed capital refers to durable producers’ goods, such as buildings, plants and machinery. In this business context, ROI is defined as the financial benefit of an investment against its associated costs. Generating an ROI metric for this type of capital is relatively easy and follows a simplified formula, which is used throughout this article.

Let’s say that to stay competitive, keep your prices stable and profits on plan, you have to increase the production of your automated plant, which operates 365 days per year, by 10 percent. Your current production output is 500 units per day at a unit value of $150. You need to add an additional 50 units or 18,250 (50 x 365 days) more units to your yearly output, yielding a value of $2,737,500 (18,250 x $150). To reach your 10 percent target, you purchase additional machines for your line at a cost of $1.3 million. After one year, the ROI on your new machines is 111 percent.

Payback Period
The next question that can be answered for your fixed capital investment is: When is my initial investment paid for? With the above figures, we can now use the formula in Figure 2, which is also used throughout this article.

You have achieved your financial goals by investing in the new machinery to increase your volume. In support of this decision, you have demonstrated a return on investment of 111 percent, and the cost of the new machinery will be paid for by the benefits of the investment in 10.8 months.

The Problem
Why can’t this same set of formulas be used to calculate ROI for training? Simply, they can – you just have to be able to overcome the main problem that Kirkpatrick has made all of us aware of. Paraphrasing the essence of Levels III and IV, “You have to see the performance before you can measure it and determine its impact on the organization.” This is not as easy as it sounds, but there are techniques that can get you as close to observed behaviors as possible without having to resort to standing behind each employee with a clipboard.

Making Performance Visible
Let’s examine three techniques that turn observed behavior into a metric that can be used to calculate ROI:

• 360-Degree or 180-Degree Evaluations
  This type of evaluation allows multiple observers (360-degree) or an employee and his manager (180-degree) to rate a common set of observed competencies, usually on a scale:
  5. Is able to coach others in skill.
  4. Can perform skill independently.
  3. Can perform skill with guidance.
  2. Conceptually understands the skill.
  1. Little ability in the skill.
  0. No ability in the skill.

  This type of instrument can be used to establish a baseline skill set prior to training and can be used after the training to determine the impact on the individual’s actual job performance. In order to generate a metric that can be used for an ROI calculation, a group of supervisors who manage the position in question need to assign a value or weight to the skill. Once this has been done, any improvement in the skill can be shown as an economic benefit.

  For example, Pat, a new manager with a salary of $60,000, and his supervisor and peers, agree that, prior to a training program on leadership, he is a 3: “Can perform skill with guidance.” Six months after the training, Pat’s supervisor and his peers now see him as a 4: “Can perform skill independently.” In Pat’s organization, 50 percent of a manager’s time is to be allocated to the skill of leadership. Supervisors who manage positions like Pat’s have determined that an improvement from a 3 to a 4 on a skill that is used in 50 percent of a manager’s job gets a weight of .05. The economic benefit to the organization is $3,000 (Pat’s gross salary of $60,000 x .05). In other words, Pat is performing the work of a $63,000 manager, but at an actual salary of $60,000. Finally, the cost of Pat’s two-day leadership training program was $1,200. We now have all of the data to calculate both an ROI and the payback period. (See Figure 3.)

• Pat’s leadership training course has yielded his company an economic benefit of $1,800 ($3,000 – $1,200), an ROI of 150 percent, and the cost of the training is paid off in four days.

• Authentic Assessment
  Rather than giving the learner a test to see if she has mastered the content of a training event, a more valid way of observing the end result of the training is to use authentic assessment. This is done by having the employee produce an end product (create a P&L statement in Excel), demonstrate new skills in a role-play (negotiate a contract) or engage in a case study (develop a market analysis of XYZ brand). While labor-intensive in its scoring, authentic assessment does provide a macro environment that allows the learner to demonstrate the new skill and to answer the question, “Can I observe them doing it?”
• Forecasting
  With the data from the authentic assessment, we are now able to forecast how this skill will impact the individual’s actual job performance.

  To illustrate, let’s take the case of June, a project manager earning $55,000 per year. June has just taken a seven-hour course on Intermediate Excel. Excel is important to Jane’s job as project manager, hence the rating of 4 for “Importance of Excel on the Job.” Before training, Jane used Excel 12.2 hours per week. After her Excel course, Jane is still using Excel 12.2 hours per week, but at a new level of efficiency that is the equivalent of using Excel 16.8 hours per week. In other words, the importance of Excel to Jane’s job and her score on a real-world task (building an Excel spreadsheet) are predictors of Jane’s performance against a “normative” population of Excel users who scored the same as Jane and listed Excel as important to their jobs – the rating of 4.

  We now have all of the data to calculate the ROI of Jane’s seven hours of training, which cost Jane’s company $60, and the payback period. To do our calculation, we need to convert Jane’s salary into an hourly rate or $55,000/1,950 hours: $28.21 per hour.

  In order to calculate the true cost of the training, we are going to add the seven hours that Jane spent off of her job while training to the $60 that the company paid for the program: (7 x $28.21) + $60 = $257.47.

  To get to the economic benefit, we need to take Jane’s hourly rate and multiply it by her virtual weekly gain: $28.21 x 4.6 = $129.77. Before we calculate the ROI and payback period, let’s project Jane’s economic benefit from the training for three months, or 13 weeks: $129.77 x 13 = $1,687. We now have all of the data to do our calculations. (See Figure 4.)

Jane’s Excel course has yielded an economic benefit over three months of $1,430 ($1,687 – $257), an ROI of 556 percent, and the cost of the training is paid off in 15 days.

Expense Savings
This is an important metric, but it is not an ROI metric as ROI has been used in this article. Expense savings represent how well you have been a responsible steward of your budget. Stewardship is something that your manager sees as a given. Consequently, it should not be the only metric that you use to defend your budget. If it is, you could find yourself in the following situation: You have a $1 million training budget, and as a good steward you save $250,000. Your manager “rewards” you by telling you that your budget for next year is now $750,000.

A budgetary defense without ROI and payback data means that you are only accounting for what you have spent and not how your budget has generated additional productivity for your company and how that increased productivity has covered the cost of the training.

Final Thoughts and Resources
This article has only scratched the surface of the metrics that could be used to defend your budget. You should also consider metrics such as usage, course completions, certification tests passed on the first try, etc. If you want your budget to be seen a critical element of a company-wide strategic effort, you may also want to consider using a balanced score-card approach as a part of your overall budget strategy.

If you are interested in pursuing the topic of training metrics further, visit www.CLOmedia.com/metrics for links to sites, books and additional resources.

James L’Allier is chief learning officer and vice president, research and development, for NETg Thomson Learning. Jim has 29 years of experience in the training industry and is a pioneer in research, design, development and management of computer-based training, interactive video and multimedia projects.