Value Optimization in Learning
A Systematic Approach for Learning Organizations to Assess Needs, Effectiveness, Business Results, ROI and Profit Impact
CLO Magazine, April 2006By John Skinner, KnowledgeAdvisors
There are numerous metrics that serve specific purposes. Some of the most critical include performance metrics, operations metrics, financial metrics, and cultural metrics. As the names indicate, performance metrics deal with the actual performance of learning programs relative to a set of criteria, operations metrics describe what is going on in the organization, financial metrics catalogue the investments made, and cultural metrics tell the story about the overall organization.
By Jeffrey Berk, KnowledgeAdvisors A constant struggle every learning organization faces is optimizing value for the investment. Optimizing is decision-making based on the past for the future. There must be a process to optimizing value and it must be built into the fabric of the learning organization to ensure it is functioning as designed.
If a process exists to not only quantify value on prior investments but forecast future value the benefits are significant and include:
Surveys are easy to scale. Items on the survey should conform to the metrics of interest—and not just collect data in a vacuum. Instead, the survey should collect what is needed. Another benefit of surveys is that they can be automated, freeing up time for analysis based on the data.
- Ensures that business results are aligned with business objectives
- Significantly reduces wasted L&D expenses
- Substantially increases productivity
- Simplifies the process of tracking actual business results
- Cost-effective approach to measuring ROI
- Enables you to link learning investments to bottom-line impact
- Data-driven approach to optimizing the impact of your L&D programs
Without a systems and processes in place to optimize value, most organizations are not reaping the full benefit of their Learning and Development (L&D) investments. More importantly, by systematically finding ways to improve impact, organizations can significantly impact earnings. For example, if an organization with $1 billion in labor cost can increase productivity by just 1%, it will increase earnings by $10 million.
Value optimization enables organizations to redirect wasted or poor-performing L&D investments into higher impact programs, which will significantly impact the bottom-line.
Case Study Highlight: The award winning Defense Acquisition University (DAU) led by CLO of the Year Frank Anderson implemented a data-driven approach to analyze value.
Over the course of three years DAU was able to significantly increase output, effectiveness and business impact without any increase in funding. This is a clear example of how an organization can systematically reduce waste and redirect investments into higher impact programs.
Human Capital Contribution ModelA systematic process to valuing learning investments is referred to as the Human Capital Contribution Model or HCCM. The model is composed of 5 core components.
Business Needs Analysis is the up-front planning process used to understand the driving business factors prompting the creation of a learning and development program.
Why is this important? Because the key to a successful learning and development intervention that maximizes impact on the business begins with a formal analysis of business needs.
What’s the process? To ensure the right L&D investment is designed, developed and delivered a needs assessment should be conducted to understand the business objectives. Once identified, a knowledge and skill assessment should be conducted to examine the gap between the current state and desired state.
What are the tools? The primary tools are needs assessments, knowledge and skills assessments, pre and post tests, and competency assessments. Using learning analytics technologies to collect, store, process and report these results, greatly facilitates the process.
What are the results? Better designed learning programs and more clearly defined and measurable business objectives that can be linked to post-learning business results.
A case study is the example of CNA Insurance. The organization was designing a new learning program. A 40 question scenario-based skill assessment was used. The net result was the skill assessment helped in the development and delivery of the training to make it more targeted to the end user.
Performance Analysis is a systematic approach to redirecting poor performing investments into programs that drive on-the-job productivity.
Why is this important? Because research has shown that nearly half (50%) of all training is wasted. In financial terms, for a typical Fortune 500 company that is more than $50 million per year, more importantly, a typical Fortune 500 company could drive another $10 million in earnings if productivity were improved by 1%.
What’s the process? To ensure waste is eliminated and productivity continually improved a systematic review of key indicators is needed to pinpoint the root cause of poor performing investments. Performance analysis focuses on where the investment occurs (Which programs? Which courses? Which vendors? What customers or lines of business?). It also analyzes key attributes of activity and performance data including attendance rates, learning effectiveness, job impact, and value indicators.
What are the tools? Data sources for this analysis include LMS (learning management system) data and evaluation data. Using learning analytics technologies to collect, store, process and report these results greatly facilitates the process.
What are the results? Better information for decision-making. Future allocations of L&D resources to more value-added programs, courses, vendors, and clients or lines of will significantly impact the bottom line.
Case Study Highlight: Nextel Communications, Inc. was conducting sales training. The company hired a third party to assist in delivery of it. Within the first days of the rollout the company was able to quickly review the results of the third party instructors. The results illustrated that certain instructors were poorly performing. The results of the above helped to eliminate waste and improve quality.
Business Result Analysis is a systematic approach to connecting L&D investments to a balanced set of actual results.
Why is this important? Because you can’t manage what you don’t measure. By systematically analyzing the link between learning and business results, one can identify areas for improvement and help drive better results. In addition, by analyzing business results, validation for ongoing investments can be made.
What’s the process? HCCM™ is based on a balanced approach and being as closely aligned to financial statements as possible. After all senior management performance is almost always based on financial results.
The recommended balanced set of business results most closely aligned with financial statements are:
1. Revenue
2. Profitability
3. Productivity
This approach applies to both the public and private sector and is recommended for all major learning programs including corporate universities, leadership programs and business unit programs. When appropriate, it is also recommended that additional business results be tracked such as quality, cycle-time, employee loyalty, customer loyalty and risk mitigation.
What are the tools? Data sources for this analysis include ERP (enterprise resource planning data), financial and cost accounting data, customer satisfaction data, error rate data, CRM (customer relationship management) data, and HRIS (human resource information systems) data. Learning analytics technologies provide L&D managers with templates and wizards to analyze the key business results but is flexible to provide for customized business results analysis and then displays the analysis on interpretive and graphical scorecards and dashboards.
What are the results? A thorough and timely understanding of organizational impact. L&D becomes advisors for improved performance.
Case Study Highlight: PeopleSoft, Inc. (prior to the Oracle merger) needed to tie the investment in training to a core set of business results. The company used both evaluation data where business results were linked to training both at the conclusion of training and months later and also an automated template to link actual business results with training. The outcome was a quantification of results that showed an improvement of 20% in productivity with the training.
ROI Analysis is a process-based approach to determining the financial return on L&D investments given cost considerations.
Why is this important? Because investments should provide more positive financial value back to the organization than the resources consumed in the original investment. It is also a commonly used financial analysis to validate investment decisions.
What’s the process? To ensure L&D management focus on benefit vs. cost leveraging methodologies from experts such as Dr. Jack J. Phillips’ ROI Process helps provide a framework for ROI. The process of collecting, storing, processing, and reporting data that create a monetary cost and benefit from training are inputs to the ROI process and then isolating the benefit to training and adjusting it for bias and conservatism is part of the Guiding Principles of the ROI Process.
What are the tools? Data sources for this analysis include ERP (enterprise resource planning data), financial and cost accounting data, customer satisfaction data, error rate data, CRM (customer relationship management) data, and HRIS (human resource information systems) data. Leverage analytics technologies to wrap automation, templates and wizards around the Phillips ROI Process to make the administrative burden of ROI feasible and practical when there are limitations on resources to apply to ROI analysis.
What are the results? A credible and reliable ROI expressed as a benefit to cost ratio that when compared by learning delivery, program, vendor, client or line of business will help the L&D manager make better decisions from a dual dimensional perspective (benefit and cost). It will also validate the investment (or invalidate it in the case of a negative ROI).
Case Study Highlight: New Horizons Computer Learning Centers, Inc. was assisting a client in technology training. Management was very interested in the financial ROI.
The key focus was on estimating the change in performance before versus after the training and isolating it to training and then adjusting it for confidence and conservatism. The cost of the training was also conservative by design.
The outcome was an ROI that was positive and produced future economic value as seen through a 3.43 payback period.
Profit Impact Analysis is a data driven planning and reporting process that helps to optimize the impact of L&D investments and to connect learning to financial statements.
Why is this important? Because L&D expense is part of the calculation, these profit measures are arguably the most meaningful ways to analyze past and future impact of learning on earnings.
What’s the process? If learning intervention is effective, it will ultimately impact profit in a positive way. The financial goal of a corporate university is to increase the profit contribution of its people. That profit contribution can best be measured by taking revenue less labor costs and L&D expense, which is called The Human Capital Contribution Profit. A corporate university will be most successful when the Human Capital Contribution Profit is growing faster then the revenue, which is measured by calculating the Human Capital Contribution Margin (Human Capital Contribution Profit divided by Revenue).
What are the tools? Data sources for this analysis include financial and accounting systems. Learning analytics technologies can provide templates to guide a manager through the inputs and structured analysis to trend and track the actual to projected results.
What are the results? A sensitivity analysis tool that can help plan investments better and optimize L&D impact. It also enables learning professionals to discuss financial impact with business executives.
Case Study Highlight: KnowledgeAdvisors, Inc. a learning analytics technology company reviewed the profit impact from a major investment in a new technology platform and analyzed the training effect on the investment. The company took a financial approach to analyze this important learning and development investment.
Projections were made for the 2nd and 3rd years after the training as management set out to prove that although the training investment would be significant it would pay off in the form of future earnings that directly impact the bottom line. The second year projection using a profit impact analysis approach was a 6% improvement in profit margin. This was because the new technology platform was projected to be brought online quicker and managed more effectively as a result of the training (the human capital contribution effect) and ultimately help accelerate future sales by 5%. The 3rd year was projected to increase profitability by 3% with a 3% increase in revenue due to the training investment.
ConclusionOptimizing value in learning and development investments is not easy. However, it is not impossible. The right systematic process, technology and templates can facilitate the analysis.
In today’s world of limited money, time and personnel we face an era of accountability for all that we do. If we are to be the fiduciary of millions of dollars for learning and development, management should expect nothing less than 1) a reasonable analysis on the business needs that drive the investment, 2) the manner in which we will optimize the performance of the investment so as to lower our risk of wasted training, 3) the linkage of the investment to key business results like revenue, profitability and productivity, 4) a reasonable ROI calculation based on the isolated and adjusted benefit from training, and 5) the projected impact on future earnings.
Based on observation in other lines of business such as sales, marketing, production planning, warranty and claims etc. these analysis have been done for years for equal strategic, visible or costly investments. Learning and development organizations should leverage the best practices of other lines of business and derive roughly reasonable ways to optimize value. Business is not about precision and perfection it’s about how to provide reasonable information in a timely manner to make better decisions. Keeping this in mind will go a long way in demonstrating accountability while illustrating how the investments will generate the most for the stakeholders.
Jeffrey Berk
Vice President, Products and Strategy, KnowledgeAdvisors, a learning analytics technology company