Most learning and development measurement still starts and ends with completions, satisfaction scores, self-reported confidence and knowledge gain. These metrics offer some level of proof, but they rarely survive executive scrutiny, and for good reason: they measure activity, not impact.
Evidence of learning impact shows up concretely in observable workflow actions, decision-making and interpersonal engagements—leading signals that can move before lagging business metrics do. This article offers a practical approach to measuring behavior change with more rigor and leveraging artificial intelligence as an accelerator where it’s useful, so learning leaders can show real impact before business results fully materialize.
Why behavior is the hardest (and most important) layer to measure
No matter which measurement framework an organization uses—Kirkpatrick, Phillips, TDRp or LTEM—the pattern is the same. Basic utilization (completions) is fastest to capture and least connected to outcomes. Reaction, self-reported confidence, and knowledge gain come next. Behavior change sits farther up the framework. It takes longer to see and is harder to prove because it requires human observation or validation. That’s precisely why most organizations stop measuring there and wait for the lagging business metrics to eventually show up.
But behavior change deserves more attention, because it’s the layer where learning either becomes real or doesn’t. A completion rate tells you who has finished a course. It does not tell you whether they know what to do (knowledge), want to do it (will), can actually do it (skill) or whether the organization has created the environment for success (can), which includes the proper tools, processes and reinforcement. All four components have to be present before a new behavior shows up on the job. When results don’t appear, it’s tempting to conclude that people didn’t get enough training. Just as often, the real gap is that the environment around them never reinforced the new behavior.
This tracks closely with the distinction drawn in a recent Chief Learning Officer webinar we hosted on measuring learning velocity through activity and capability: more content, faster rollout and higher completion numbers don’t automatically produce faster capability and can even create the opposite (overload, slower decisions more hesitation).
Learning velocity, in that framing, isn’t how fast you launch training; it’s how fast the business gets measurably better at the work. And it only becomes visible through leading indicators—decision speed, confidence to act, willingness to experiment, reduced hesitation.
The maturity model behind the shift
In a live poll during the webinar, more than half of attendees said their organization still primarily tracks completions and participation. Why is that? Our “2025 Measuring the Business Impact of Learning Report” sheds some light on why completions and participation remain the dominant reporting metrics:
- 40 percent cite competing priorities as the primary barrier.
- 20 percent cite capability gaps in measurement skills.
- 17 percent cite data access challenges.
The pattern holds industrywide: Organizations aspire to measure beyond completion rates, but lack the resources, systems or operating model to do it consistently. We recommend using a measurement maturity model to assess a measurement practice across three categories: people, technology and process, each progressing from “emerging” to “mature.”
On the emerging end:
- Measurement lives with isolated specialists who have limited capacity and no real ownership beyond ad hoc advocacy.
- Standard LMS reporting and spreadsheets are built on traditional learning and utilization data analyzed manually.
- Processes around the measurement practice have little governance: Measurement gets designed around compliance or a learning objective, executed as a default set of learning metrics and used reactively, if at all.
On the mature end:
- A strategic leader owns the measurement practice, has resources that extend across the enterprise and is supported by specialists who are externally recognized thought leaders.
- Integrated and automated people, learning and business data enable AI to move past descriptive reporting into exploratory and predictive analytics that enable personalized learning.
- Full governance and standardization are in place, with L&D and the business collaboratively planning measurement initiatives annually.
Most organizations are on the emerging end of the spectrum. That’s not a criticism; it’s the reason this piece exists.
AI as an accelerator for measuring behavior, not just for delivering it
Behavior has always been hard to measure because someone had to watch it happen, like a manager running down an observation checklist or a coach sitting in on a call. Both approaches are slow, inconsistent and expensive to scale. AI now makes it possible to observe behavior at a scale no human team could match.
One practical example: a prompt to evaluate executive presence built for project managers. After completing training, which included a rubric for what strong executive communication looks like, participants could feed a recording and transcript of an actual meeting into any AI chat tool, along with the prompt. The AI evaluated communication style, message structure, audience engagement and executive presence against that same rubric, then returned individualized feedback.
What made the experience especially valuable was that it didn’t stop at the feedback. Participants could continue the conversation, asking questions tailored to their own learning needs, such as, ”How could I have phrased that differently?” Or, “What would executive-level communication sound like here?”
The result was personalized reflection, practice and skill-building. At the end of the engagement, the prompt would include a link to a brief survey that captured relevance and key takeaways, providing an additional source of leading-indicator data.
Nothing about this required a licensed platform, a custom model, or a vendor relationship. It’s a single, detailed prompt that can be copied, adapted and reused across teams. What it produces is the kind of leading-indicator data that used to be out of reach: A scalable, repeatable read on an interpersonal interaction that once required a human observer, supplemented by learner-generated insights on what was most relevant to their work.
Building measurement around the rhythm of the business
A second example, from a new-manager onboarding redesign for a large enterprise client, shows how far this approach can go. In place of a standard sequence of onboarding modules, we recommended an AI-enabled facilitator sit on top of the onboarding content and support each new manager throughout roughly their first 18 months.
In addition to answering questions associated with the onboarding content, the facilitator connects to HR data, so it’s aware of the manager’s start date, direct reports, and the organization’s “rhythm of the business,” or ROB (i.e., quarterly conversations, promotion cycles, budgeting discussions). It can nudge managers ahead of those moments of need, even providing opportunities to practice via AI-role plays aligned to the ROB critical conversations.
That timing detail matters: The old 21-days-to-build-a-habit rule assumes daily repetition, but a quarterly conversation only happens once a quarter, so reinforcement has to follow the actual cadence of the work, not a generic time-to-competency benchmark.
Designing measurement around the rhythm of the business is what lets this program track two different kinds of signals at once: the workflow action of engaging with the AI facilitator, and the decision-making it was meant to improve—whether a manager walked into a promotion conversation or a budgeting cycle prepared. In this scenario, some leading indicator metrics to watch would be:
- Early engagement with the AI facilitator: Are we gaining momentum? If not, how should we shift, revise or adjust?
- Relevance and satisfaction metrics: This data is a quick way to determine whether what was provided was relevant and sufficient.
- Reduction in HR inquiries: A 25 percent reduction in questions from new managers related to the rhythm of the business and “where do I go” would be a good target.
- AI roleplay data: Percentage successfully demonstrated behaviors and trends in qualitative feedback can reveal how accurately and consistently new behaviors may show up.
- Team and manager-of-manager pulse checks: Brief, recurring checks around observable behavior at 30, 60 and 90 days.
An honest complication and where AI changes the equation
Not every leading indicator is as directly observable as a recorded sales call or AI roleplay. Several of the signals that matter most, like confidence to act and willingness to experiment, are typically captured through self-reporting, which is vulnerable to halo bias. People rate themselves more favorably than their behavior would support, especially soon after training.
Webinar attendees raised this directly, along with a related and harder question: How do you draw a straight line from a learning initiative to an organizational outcome when promotions, retention, and performance are also shaped by coaching, mentorship and stretch assignments that have nothing to do with a course?
There isn’t a clean answer to either question yet, and this piece won’t pretend there is one. But the AI-enabled examples above point to real progress on the self-report side.
When AI can evaluate an actual behavior—a meeting, a piece of written work, a workflow action—the resulting leading indicator is no longer a self-rating. It’s a direct read on what happened. And it’s generated at scale no manager, coach or observation checklist could ever sustain. It won’t close the attribution gap, but this moves the needle away from relying on how people rate themselves and toward what AI can actually observe en masse for the metrics that matter most.
