Estimate the Impact Before Investing In Training
PerformanceXpress (an ISPI publication)
Maximizing the performance of your human capital is an advantage if leveraged. Training is a tool that if applied, can certainly be a catalyst for maximizing human performance. One way to do this is by effective measurement.
The purpose of measurement is to derive a process whereby you can estimate the change in human performance, isolate it to a driver of human performance such as training, and make adjustments for conservatism.
Estimation is a process commonly used in business today. Sales people will estimate their future sales, accounting people will estimate the cost of a warranty or claim that is expected in the future. So too can training personnel ask that participants, supervisors, experts and others estimate the job performance impact that a training program will have on the job. Participant estimation, as it is commonly referred, is not estimating the performance solely related to training but asks participants to estimate job performance changes in general, including among other factors, training.
For example, if one attends sales training, one might estimate an increase in job performance but that increase could be related to other factors such as a competitor going out of business. So, estimates of performance change need to take into account many factors. Those factors include process changes, people changes, marketplace changes, technology changes and interventions like training.
When estimating the increase, the person(s) doing the estimate should think carefully about all the factors mentioned. They may want to review historic data and forecast data to reasonably factor into their overall performance change. In addition, they may want to look at business results such as quality increases, sales increases, cycle-time decreases, cost decreases, risk decreases, etc. (the end outputs of human performance change) before vs. after training and compared to a control group who did not receive the training.
Logically, the training department is keenly interested in the effect training had on the performance improvement. So, the next step is to isolate the estimated increase in performance to just training. In this part of the process, the person(s) doing the estimates should estimate how much the training has or will influence job performance, relative to the other factors and assign a value to it. So if the sales person felt that training was the strongest factor that caused change or will be the driving force behind future change it would receive a higher value than not.
Finally, because estimation and isolation is subjective at times, one must adjust any results for the estimate. Again, in other facets of business this is commonly done. Using analysis such as most likely, optimistic and pessimistic adjusts estimates for bias by the estimator and flaws in assumptions. You’ll often see sales forecasts reported in this manner.
In training, adjustment is made for two reasons: first is conservatism.It is also critical to state one is conservative in assumptions to build integrity into your metrics. Second, is for bias. Estimates can be inflated. In fact, studies done by organizations like the Tennessee Valley Authority (TVA) and separate studies by KnowledgeAdvisors suggest that respondents tend to over estimate by a factor of 35%. To this end, when computing a human performance change one might reduce the inputs by a factor of 35% or a similar confidence rate as the adjustment factor for conservatism and bias.
Taken together, the principles of estimation, isolation, and adjustment form a powerful model in tabulating a systematic, replicable, and comparable model for human performance change.
A reader may say ‘this is not credible data, it is not statistically accurate, it is too subjective.’ My response would be that that the world of human performance measurement is far from objective and accurate. The goal is to have roughly reasonable indicators of it without expending considerable human, financial or physical resources to do it.
Recognize that your attempts to go from roughly reasonable to highly accurate are tremendous outlays in resources. In addition research states that you should do otherwise. First, a study published in the May 2003 Harvard Business Review found that senior managers make decisions on other instinctive factors not the highly accurate and highly costly data they are provided from highly paid number crunchers. Second, they use such data as one of many inputs and prefer more timely rough estimates versus precise metrics that are too late to factor into decision-making.
Conclusion:
The first step in human performance measurement is to understand that in a world of doing more with less, any element of the organization that is involved with performance improvement needs to be seriously thinking about how to measure the impact their initiatives have on human performance.
About the author:
Jeffrey A. Berk
jberk@knowledgeadvisors.com
Jeffrey A. Berk is Vice President of Products and Strategy for KnowledgeAdvisors. KnowledgeAdvisors is a corporate learning business intelligence firm that helps organizations gain the knowledge to improve human performance, better educate its workforce and reduce costs across the enterprise. Its proprietary measurement technologies and benchmarking expertise help companies more successfully measure human performance change due to training.