The structure of a course review depends on the QM Rubric you select, the type of course you want to evaluate, and the type of review you want to conduct. For example, a Self-Review, Internal Review, and Official Review will all be structured differently. And the QM Rubric you select for your organization will affect the composition of your review team, as well as the QM Standards you have to meet.

Organizations that are just starting out with QM would encourage faculty and staff to conduct a Self-Review. This is an informal process that helps individuals get comfortable using QM Standards to privately evaluate one of their own courses. An Internal Review is also informal, but it is a planned, ongoing, and collaborative peer review process that enables organizations to get specific and constructive feedback that can be used to inform improvement of course design. This process allows organizations to prepare their courses for an Official Course Review—in the hopes of meeting QM Standards and achieving QM Certification.

With the exception of a Self-Review, all other reviews begin with the submission of a Course Review Application. This includes identifying the organization’s contact person (QM Coordinator or QMC), the course contact person (Course Representative), the type of review (Internal, QM-Managed, or Subscriber-Managed), and basic course information. Anyone within the organization can submit a Course Review Application.

Based on the type of review, a QM staff member or the QMC is responsible for reviewing and approving the application. When the application is approved, the Course Representative (instructor) is notified by email that the Course Worksheet is available in the system for completion. The Course Representative uses the Course Worksheet to provide general information about the course for the review team to use as they're reviewing a course. Several Rubric Standards also contain annotations that direct reviewers to look for information supplied on this Worksheet.

Concurrently, the QM staff member or QMC identifies possible review team members and contacts each one to confirm availability and interest in serving on the review team. The composition of the review team* depends on the QM Rubric you select. However, for reviews that lead to QM Certification, all reviewers must be QM-Certified Peer Reviewers.

• A Higher Education (HE) and K-12 review team needs at least one reviewer who is external to the organization submitting the course, at least one reviewer who is a Subject Matter Expert, and a Team Chair who is a QM Master Reviewer.
• HE Publisher review teams need two HE Publisher Peer Reviewers, where both are Subject Matter Experts, and a Team Chair who is a Master Reviewer.
• K-12 Publisher review teams need two K-12 Publisher Course Reviewers, where one is a Subject Matter Expert, and a Team Chair who is a Master Reviewer.
• A CPE review team needs two CPE Reviewers, where one is a Subject Matter Expert, and a Team Chair who is a Master Reviewer.

After the QM staff member or QMC adds the review team, each reviewer has access to the Course Worksheet and their individual Reviewer Worksheets. Once the review is opened, the Chair sends an Introduction email, which also requests course access and scheduling of the Pre-Review call. The Pre-Review call allows the instructor to "meet" the review team and answer any questions before they start their review. Prior to the call, the review team should review the Course Worksheet and log into the course to see if they have any questions. While we recommend that all reviewers attend the Pre-Review call, we require the Chair to have this call with the Course Representative. Typically this call takes about 20-30 minutes.

Following the Pre-Review call, the review team uses the Course Review Management System (the fully automated rubric tool) to complete their independent reviews. The feedback provided in a course review is collegial in tone and language; however, the reviews are rigorous and even courses that meet Standards will benefit from the detailed, specific, and relevant feedback that is provided.

The reviewers independently look at each Specific Review Standard and decide whether it is "Met" or "Not Met," applying an 85% threshold in making his or her decision. Each reviewer is required to provide evidence to support his or her score.  If a reviewer finds that a Standard is "Not Met," he or she is also required to provide suggestions for improvement that include specific changes that a course developer should implement for the course to meet the Standard. For Standards that are "Met," suggestions for improvement are not required, but suggested as a good practice. During the review, any questions about the course are directed to the Chair, who then contacts the Course Representative.

In order for a course to earn QM Certification through an official Subscriber or QM-Managed review, it must meet all Essential Standards and receive a minimum score, which is specific to the Rubric used for the review.

At the conclusion of the independent review, the review team conducts a Post-Review call to discuss the review and any Standards that are not met. The Course Representative from your organization is not included in this call. When ready, each reviewer submits their Reviewer Worksheet. The Chair then submits the Final Report. The Final Report shows the aggregate findings of the reviewers for each Standard. It includes their determination of "Met" or "Not Met," as well as the recommendations made by each reviewer.

A typical Official Course Review takes 4-6 weeks. If the course does not initially meet QM Rubric Standards, you can submit an amended course within 14 weeks of the date the Final Report was submitted. The Team Chair will then review the amendments and confirm if the course meets QM review standards. If the course has met QM Standards, the course will achieve the QM Certification Mark and be recognized on the QM website. This review process is intended to help organizations create a culture that is focused on quality assurance and continuous improvement of their online and blended courses.

*Note: A single person could play multiple roles. For example, one person could be both the SME and the Master Reviewer All teams still must have the specified number of members, based on the type of review.