Training and accountability
Once upon a booming economy, training programs enjoyed a unique status in the business world as expenses that were typically as easy to justify as they were impossible to evaluate. Like mission statements, company baseball teams and the art on the conference room walls, they existed and they were good, even if it was tough to explain exactly why.
Training programs still exist today, of course, but their corresponding automatic assumption of worth is fading like a shadow in the gloomy light of the current downturn. Companies have to be accountable now for the money they spend on everything, even training.
To complicate things still further, just as training programs are being subjected to greater scrutiny, organizations are becoming increasingly conscious of the importance of knowledge, of learning, to their entire enterprise. The added effect of this new awareness is to turn up the pressure for training accountability even higher. This dual dilemma has many managers looking for ways to access the worth of their existing initiatives as well as the potential return on any investment in new programs — training and otherwise.
GOOD FOR BUSINESS
“Good” in today's business environment means, first and foremost, good for the bottom line. In the case of training, however, even companies that have attempted to assign value to employee education programs have tended to use other measures. One of the most popular evaluation systems, for instance, is still Donald Kirkpatrick's “four-levels” approach, first introduced in a series of articles in 1959-60 and later collected and published together (Kirkpatrick, Donald. Techniques for Evaluating Training Programs. Alexandria, Virginia: American Society of Training Directors (ASTD), 1975).
Kirkpatrick described four questions or “levels” of training evaluation:
-
Level 1, Reaction: How do the trainees feel about the program?
-
Level 2, Learning: What exactly did they learn?
-
Level 3, Behavior: How did trainee behavior change based on what was learned?
-
Level 4, Results: Did the training have a positive impact on the organization?
While this is a very useful way to begin to consider training outcomes, it does not by any means cut straight to the bottom line. Even the “results” level is still at arm's length from ROI, stopping short of a true cost-to-benefit assessment. The other problem, of course, is that the approach really only works after the fact and can't be used to help guide decisions about new training beyond saying, “We'll never do that again!”
ON THE ROI RADAR SCREEN
In order to both plan for training and determine its impact on the bottom line, companies today might do well to borrow techniques from other disciplines, such as information technology and finance. Consider Robert Benson's system, called “The Profit Model,” for example. It was originally developed by Benson and his colleagues to help IT executives talk about technology implementation proposals in terms of profit potential rather than bits, bites and protocols, but it sheds a strong new light on training programs, as well.
“Many CIOs and other technology managers [read training managers] have had a two-fold problem,” Benson notes. “First of all, they have often worked in a vacuum, isolated from the strategic business information that should be directing their efforts, either because no clear strategy exists or because they are not a party to it. Secondly, they have not been effective in communicating the ways in which technology [training] can improve performance to impact the bottom line.”
Benson suggests that managers must be able to answer three key questions:
-
What are the company's “profit drivers”? That is, what business activities create profit either directly or indirectly, and how do we measure performance in those areas?
-
What does senior management intend to do to improve the performance of profit drivers?
-
What can a given initiative do to support management's intentions?
The instant a company starts to view training (or any other business function) in terms of its impact on profit drivers, programs begin to lose their vague, amorphous shapes and assume the critical weight and mass necessary to get them on the ROI radar screen.
Because profit drivers are forward-looking indicators, leading rather than trailing changes to the bottom line, this is an especially valuable perspective shift. It not only helps companies to determine if training is changing behaviors, it allows companies to look ahead with more confidence and project the potential impact of a given initiative on profit. The methodology can even be used to help evaluate programs under consideration by creating “what-if” scenarios and looking at their probable impact.
Truck fleets have several profit drivers or performance indicators that they might choose to track, such as accounts retained or lost per quarter, new accounts per number of sales calls, number of customer complaints, errors per 100 invoices, percentage of on-time and late deliveries, out of route miles per month, cycle times, employee turnover or absenteeism. These indicators are not directly expressed in red or black ink, but changes to them are advance signals of potential changes to the bottom line, which is what makes them such useful metrics.
This means that calculating the ROI for a given training program, for instance, is really a three-step process. The first step is selecting the profit drivers you want to track. Step two is establishing a cause and effect relationship between the program and changes to the profit drivers and, finally, step three is assigning a dollar value to those changes.
Inevitably, in the case of training, conscientious managers find themselves wondering, “Should I be calculating the ROI for every program, regardless of scope and cost? Is it worth all the effort?”
According to many experts, like Jack Phillips (Phillips, Jack. Accountability in Human Resource Management. Houston, Texas: Gulf Publishing Company, 1996) the answer is no. After all, some training is government-mandated, so it is a must-do regardless of the cost. The same is true for education on things like a new routing and tracking system. If you are going to deploy it, then people simply have to learn how to use it. Still other training, like a month-long Yoga for Health session offered to employees after work, probably just does not merit the effort required to gauge its effect on the organization as a whole. Instead, experts like Phillips recommend that companies focus on calculating the return on investment only for ongoing training initiatives (such as in-house driver education and sales training programs) and for very expensive or very critical programs with a high potential impact on the organization.
This kind of holistic, enterprise-wide perspective is not new for many functions within business. Companies have long scrutinized activities such as new product development, mergers and acquisitions or key management changes, for instance, in the context of their potential effects on the future of the organization. Now the insistence on accountability has finally also reached training's door.
It is good news and it is high time. Without evaluating training and understanding its impact, companies cannot effectively employ it to help shape the vibrant, knowledge-based organizations of tomorrow.
Truckload Carriers Association
2200 Mill Road ▪ Alexandria, VA 22314
Telephone: 703-838-1950
TRAINING DATA
Training is big business. Companies in the United States budgeted a total of $54 billion for formal training in 2000, according to Training Magazine's “Industry Report 2000.” In fact, the American Society for Training and Development (ASTD) recently reported that the average U.S. company is training record levels of employees, 78.6%, and that the top 10% of companies surveyed are training virtually every employee — 98.4% of their work force. These top firms also reported spending an average of $1,665 on training per eligible employee, compared to a figure of $677 per employee for average companies.
Are companies also assessing the value of these major financial commitments?
“Training, like any other financial investment, is not immune to changing times,” noted Mark Van Buren, director of research for ASTD in the organization's “2001 State of the Industry” report. “Now, more than ever, companies must continually demonstrate the value and worth of their investments in training.”
Training at Sure Bet Trucking, Inc.
Contrary to their name, Sure Bet Trucking, Inc. has been losing shippers at an alarming rate so COO Frank Kincade has decided to implement a training program for truck operators and fleet managers called “Working Together for Success.” The big question is: How can they evaluate the new training program early on to be sure it is not just a waste of time and money?
Since the defecting accounts have been citing trouble with late deliveries, issues with the steady stream of new drivers on their account, and cargo damage as the causes for changing carriers, Kincade chooses to track number of late deliveries, driver turnover on key accounts and number of damage claims per month. He wants to evaluate only the impact of the new training program, so he also elects to delay some other actions, like implementing the new invoicing system, until the results of the training can be measured.
The baseline data for his selected profit drivers during the 12-month period prior to training are pretty gloomy, but the worried COO plunges resolutely ahead:
Driver turnover on key accounts:
90% (or 90 truck operators out of the 100 total)
Average damage claims per month: 15
Average late deliveries per month: 18
Six months after the training, things are beginning to look up, so Kincade is eager to do some annualized ROI projections to share with the stockholders and the CEO.
Driver turnover on key accounts:
10% (or 10 truck operators out of the 100 total)
Average damage claims per month: 5
Average late deliveries per month: 10
Encouraged, Kincade starts rolling up the numbers tied to these improvements:
Driver turnover:
$3,500 total per driver recruiting/replacement costs (or a projected annual savings of $280,000)
Damage claims:
10 less per month × $275 per average (or a projected annual savings of $33,000)
Late deliveries:
8 fewer late deliveries per month × $500 average total penalty cost (or an annual savings of $48,000)
When he totals up Sure Bet's savings, Kincade is pleased: $361,000. The training, however, was not free, so he has to subtract the costs associated with it: The tuition was $250 per trainee or $25,000 for the 100 drivers plus an additional $1,250 for their four regular dispatchers and one back-up person. Then there was the cost of those lunches — $5 per person per day, or $1,575.
Time away from work costs money, too. Drivers missed eight hours per day for three days at an average wage of $11.65 per hour; the four regular dispatchers missed eight hours per day at an average wage of $14.10 per hour for the full-timers and $10.15 per hour for the part-time back-up person, making the total cost for time away from the job $27,960 for the drivers and $1,597.20 for the dispatchers, or $29,557.20. This means the actual total cost for the training was $57,382.20.
So, was the training worth it? Kincade calculates the ROI based on the data he has tracked:
Net gain = (total benefit-total cost)/total costs = ___ × 100 = ___
Or in Sure Bet's case:
Net gain=$361,000 - $57,382/$57,382 = $303,681/$57,382 = 5.29 × 100 = 529% ROI
As Kincade is making up his Power Point presentation for the senior management meeting, he realizes that, as good as the ROI is, he has not factored in a long list of other benefits. For example, did the number of lost accounts per month drop? What about customer satisfaction at existing accounts and any possible incremental revenue? Employee morale as it relates to absenteeism and errors on the job? Increased productivity?
While Kincade has probably under-valued the benefit of the new training program, he has accomplished the real purpose for his evaluation. First of all, he knows that the new program is actually helping to correct some serious problems within the business. More importantly, however, by choosing to look at the impact of training on some key profit drivers, he can predict with some confidence that the training should have a positive impact on the company's bottom line.
