We at the Graduate School of Education at the University of Pennsylvania are delighted to work with Chief Learning Officer magazine to bring you the best scholarship from our doctoral students and faculty. The focus of this column is to present the most innovative research emerging from a unique group of leaders who are concurrently scholars.
To kick it off, I’m going to talk about one of my favorite topics: “research” that people consume without question and myths that seem to defy evidence. As far as I can tell, the best-selling authors and biggest gurus are all salespeople trying to peddle their goods. It doesn’t mean they aren’t honest, decent people with value, but they have a vested interest in seeing you purchase what they are selling.
The news isn’t all bad. The theory of human capital development suggests that if we develop people, they will become more productive. The problem is, empirical research suggests between 66 and 80 percent of the variance in performance is not captured by human capital development models. At best, we are able to impact 34 percent of the performance variance. And yet, the space seems to operate like learning is an elixir, curing any ill.
The preponderance of evidence around leadership development suggests that if you’re doing it to improve leaders’ performance, you’re probably deluding yourself in terms of efficacy. This doesn’t mean some of you aren’t impacting some leaders. There is also evidence that leadership development programs do other things, but our literature review found that only 7 percent of the studies looked at leader performance and none looked at team or organizational performance.
Now that you’re mad at me, let me share some other trendy concepts from corporate learning that empirical research suggests are problematic. For each of these, a team of doctoral students and I spent several months chasing down the research, which indicated the proverbial emperor has no clothes.
Informal learning: 70-20-10. Does informal learning exist? Folks generally would say yes. But where did the ratio come from and what does it mean? It basically comes from our old Greek friend Archimedes. Then there was an academic, Allen Tough, who in the 1960s used the iceberg as metaphor, and somehow it got popularized as being a formula. Archimedes demonstrated why most of the iceberg is below the surface, and that is where the numbers come from. Think about it: what is the ratio? Is it hours? Compensation? How was it empirically studied? It’s like that old game — telephone. Someone took a conceptual metaphor and then made a misguided inference.
Learning styles: A review of about 150 studies found that none supported learning styles. The mind is so complex and malleable that variance within a person and between people is so great as to make the point moot. Think about it intuitively: if you were a visual learner and were blinded, do you think you’d stop learning? Poor reading of popularized neuroscience books is further obfuscating thinking.
ROI: As far as I can tell, this is a conceptual metaphor for “Was it worth it?,” which is reasonable. But real cost/benefit analysis such as net present value is complicated. It’s been done successfully since 1901 when the Army Corps of Engineers introduced cost/benefit analysis, but among other things, you’d need to figure out all the benefits and costs, monetize them, figure out a discount rate and control for other factors that might influence organizational bottom line performance. James Heckman won the Nobel Prize around some of these things and found that investing in training pays dividends, around 10 to 20 percent. At corporate learning conferences I’ve heard claims that approach 100-fold on such returns. It’s a crock.
I’ll share more insights in upcoming columns on common learning theories that may have a few holes.
Doug Lynch is vice dean of the Graduate School of Education at the University of Pennsylvania and created the first doctoral program for corporate training, the Penn CLO Program. He can be reached at editor@CLOmedia.com.
