Research on the impact of Quality Matters has been conducted since 2005. The results of this research — with all of its nuances — are compiled in the Research Inputs and Outputs of Quality Matters: Update to 2012 and 2014 versions of What We’re Learning from QM-Focused Research by QM Director of Research Dr. Kay Shattuck. This page is a brief discussion of QM’s impact.
What Is Important About the QM Rubric?
The QM Rubric and process are supported by research (see the QM Research Library) that establishes the importance of a variety of features in constituting a quality online course and the importance of quality course design as a component of a quality online course. The Online Quality Pie, above, identifies the other critical factors affecting online course quality. Institutions and organizations using QM as an integral part of their online quality process, along with the other critical elements of online course quality, positively impact their instructors and staff, their learners, and their culture to one of collaboration and continuous improvement.
What Is Important About QM Implementation?
The motives for subscribing to Quality Matters are many and varied. Implementation of QM tools, resources, and processes can be understood as an institution's steps toward excellence beginning with an ad-hoc approach in which individual faculty or course designers use the QM Rubric and professional development resources to improve selected courses. Steps toward excellence* continue toward systemic institutional change with embedded processes. Any research project focused on the impact of QM needs to identify the implementation level of QM at the institution.
Examples of QM Impact
Official QM Course Reviews - both QM- and subscriber-managed - result in all participants (Course Representatives and Review Team members) taking the recommendations and experience from the review back to their own online and face-to-face courses and improving them. For Course Reviews completed between February 2015 and October 2016:
- 94% of Course Representatives and 81% of Peer Reviewers indicated they either had or planned to make changes in their online courses as a result of their Course Review experience
- 80% and 65% of them, respectively, indicated they either had or planned to make changes in their face-to-face courses
The learning that takes place in QM Professional Development results in participants taking that knowledge and applying it to key aspects of their own online and face-to-face courses. According to 2016 research done by Lorna Kearns and Rae Mancilla at the University of Pittsburgh in 2015, participants in QM Professional Development from 2012 to 2015:
- Made changes in their online, blended, and face-to-face courses as a result of their experience, including revising their learning objectives, improving their course alignment, and paying greater attention to their communication with students
- 94% of participants in Kearns and Mancilla’s study indicated they made changes to their courses as a result of participation in QM PD; 3% indicated the question was not applicable
Students Impacted by Course Reviews for the Sixth Edition of the Higher Education Rubric
13,122 QM-Managed and Subscriber-Manager Course Reviews
1 Course Review impacts 360 students:
1 course x 4 review-participating instructors x 3 courses x 30 students
13,122 reviews x 360 students = More than 4.7 million students impacted
What Is Your Quality Goal?
From satisfied learners who have what they need to successfully complete courses to increased course completion and student learning, each institution or organization has its own priorities for increasing the quality of its online courses and programs. QM is an important component in a comprehensive approach to quality online education. To track progress towards quality goals, how — and the extent to which — QM is used, must be assessed within the context of the other variables that have an effect (i.e., the Online Quality Pie) on the outcomes. For use of QM that goes beyond the ad hoc approach toward institutionally embedded policies and processes, this framework provides guidance on setting up a study that assesses QM impact:
Prior to embarking on a study, it should be kept in mind that all educational research is challenging, primarily because of ethical and practical constraints. Any search for evidence of the impact of QM must deal with those challenges, plus use strategies to identify and isolate specific QM features and processes tracked during a study. For example, a study assuming that the event of a QM formal review of a course would provide simple, direct evidence of the “impact of QM” is fraught with validity concerns if the study does not include an analysis of the preconditions of the course design and documentation of modifications made as a result of the formal review. The situation before the QM “treatment” needs to be acknowledged in order to make any conclusions about the impact of QM. Additionally, the possible impact of other factors known to affect a quality online course (see the Online Quality Pie), need to be correlated or at least acknowledged, as does the level of QM implementation.
*Steps toward excellence model and graphic developed by Deb Adair, 2016
