Roosevelt University Evaluates its Fully Online Program with Metrics that Matter®
Roosevelt University Evaluates its Fully Online Program with Metrics that Matter®
Description of Learning Program: Roosevelt University offers a wide variety of undergraduate, masters, and doctoral programs. Schools within Roosevelt University include: Arts and Sciences, Walter E. Heller College of Business Administration, Education, Evelyn T. Stone University College, and The Chicago College of Performing Arts. RU Online, Roosevelt University’s fully online learning program, delivers certificates, degrees, and individual courses to students in the Chicagoland area as well as across the country.
Measurement Approach: Prior to implementing Metrics that Matter® Roosevelt University’s RU Online Program had begun to use a web-based surveying tool to survey students and faculty at the mid point and at the end of the semester. While the tool was easy to administer, the reporting capabilities were limited, requiring RU Online staff to conduct extensive offline analysis to view the data in the manner they required.
Roosevelt University’s RU Online Program engaged KnowledgeAdvisors for a pilot of the Metrics that Matter® system. Several reasons for choosing KnowledgeAdvisors existed including the following:
- Tap into learning measurement experts to review and improve existing evaluation tools
- Leverage more specific learning measurement software to streamline the data collection function
- Exploit the reports that accompany the Metrics that Matter® system to streamline offline reporting and analysis
- Provide a more timely self-service internet model to allow key stakeholders (administration and faculty) to view tabulated and aggregated results quickly
- Place into practice a more practical, scaleable, and replicable solution for ongoing student and faculty evaluation processes
KnowledgeAdvisors experts worked with RU Online staff to edit existing instruments. KnowledgeAdvisors then setup a Metrics that Matter® account for Roosevelt that included all of the fully online courses the University wanted to measure during the pilot. Data was then collected via the Internet at the mid point of a term and then again at the end of the term also picking up data from the faculty. The data was then stored, processed, and reported in Metrics that Matter® in a real-time manner and accessible to administration and faculty with account access.
Measurement Obstacles: Proving the value of Metrics that Matter® over the existing online process was a challenge. However, the pilot program was an essential partnership between Roosevelt and KnowledgeAdvisors to prove the concept and value. The pilot clearly demonstrated value even after the mid-term data was collected and the RU Online staff and faculty could immediately see aggregated results and qualitative comments.
Measurement Successes: As a result of a healthy collaboration between Roosevelt University and KnowledgeAdvisors, the pilot program has proven to be a valuable exercise that demonstrated the impact learning measurement software can have in an academic environment.
RU Online administrators can search the system to find the top and bottom courses, curricula and programs in easy-to-view drill down reports. They can quickly determine the rank order of faculty based on student evaluation responses, and they can analyze faculty feedback specific to each course to make changes for future courses. Roosevelt also has access to a benchmark database with over 46 million data points to compare its offerings to others who use Metrics that Matter®. Future plans for Metrics that Matter® include expanding the use of it beyond the pilot programs and continuing to improve their services and demonstrate value resulting from the measurements. This is made possible because less time is now spent deriving the measures so more time can be spent analyzing information and taking action based on the measures.