Microsoft Leverages Metrics that Matter® to Manage Its Certified Learning Channel
Company Overview: Microsoft is the worldwide leader in software, services and Internet technologies for personal and business computing. The company offers a wide range of products and services designed to empower people through great software—any time, any place and on any device.
Description of Learning Program: Microsoft partners with third party organizations to conduct training and certification on its software products. Microsoft engages hundreds of these professional learning providers (known as Certified Partners for Learning Solutions or CPLS) to train thousands of individuals every year on Microsoft Official Curriculum (MOC).
Business Objectives of the Program: Microsoft Learning offers comprehensive training solutions to their customers to help them build technical solutions and to stay competitive today and in the future. Microsoft Learning uses premier learning solutions partners called CPLS to help its clients develop expertise on the latest technologies through instructor-led training, e-learning and blended learning offerings. Microsoft must ensure that the learning providers are of the highest quality when carrying out Microsoft training. To Microsoft, tracking customer satisfaction and quality of the third party vendors is critical. In addition, training is a key indicator of product and end customer business intelligence. The need for accurate and timely knowledge of the products being trained upon and the organizations taking the training can provide Microsoft with essential knowledge on product and customer demand. Microsoft is adding to its base of performance measurement by tracking a balanced scorecard of all five levels of learning, including ROI. This will help the organization pinpoint how the software and the related training are impacting people on the job, how training ties to organizational business results, and if the training was a positive investment for the end customer.
Measurement Approach: The measurement approach was complex and had been attempted before by Microsoft in the past but due to logistical, cultural and technical challenges had not been successful. Then, in 2001, Microsoft Learning and KnowledgeAdvisors teamed up to attempt to measure in a consistent, real-time, and on-line manner, the training performance of every participant taking Microsoft training from all certified vendors. Microsoft utilized the power of KnowledgeAdvisors Metrics that Matter® system to do this. Metrics that Matter® is able to present the right survey, at the right time, to the right participant thousands of times per day to collect, process and report the results of hundreds of training events around the world instantaneously. Use of Metrics that Matter® saves Microsoft significant financial, physical and human resources that would otherwise be expended managing such a large and complex undertaking in the absence of Metrics that Matter®. Participants are surveyed at the end of training and when they are back on the job. The current program captures additional business intelligence on key metrics surrounding learning effectiveness, job impact, business results, ROI, and product and marketing information.
Measurement Obstacles: Unlike a corporation that controls its destiny as it produces and delivers the training, Microsoft had hundreds of independent learning providers that deliver the training on its behalf. A major challenge was convincing the learning providers to use a tool like Metrics that Matter® and to be measured and benchmarked against each other. This was overcome because a value-added feature of Metrics that Matter® is in its ability to automatically provide over 100 reports online to each individual learning provider, to all of its locations, and to all of its instructors. In this way, the learning providers received value from the process in a tool that helped them manage their business better and prove value to their end clients as well.
Measurement Successes: To date, millions of data points on hundreds of thousands of evaluations have been collected for Microsoft certified training programs using Metrics that Matter®. Microsoft has access to information that ranks each certified vendor in terms of several indicators of performance. In addition, Microsoft generates ROI information that can be analyzed in a real-time manner. Information regarding job impact, business impact, and a financial return on investment are reviewed by Microsoft on a monthly basis and can be sliced and diced by program, curricula, course, location, learning provider, client, learning delivery etc.
Based on the success with the CPLS program, Microsoft has now rolled out Metrics that Matter® with their Partner Program and their internal learning organization.