What Does a QA Comprehensive or Sample Review Cover?

What Does a QA Comprehensive or Sample Review Cover?

Overview

The QA team performs thorough course design and implementation reviews on all or a sample of courses in an established program track. Reviews are conducted in collaboration with the partner school. These in-depth course reviews include an evaluation of:
  1. Course design and development
  2. Time on task
  3. User experience 
  4. Implementation in the LMS
  5. ADA compliance
  6. RSI (regular and substantive interaction) compliance
  7. Documentation and resources for instructors
  8. Student feedback
  9. Student performance data
Each course is evaluated on the set of criteria on the QA score sheet using the Course Design QA Rubric. The Orbis Course Design & Development QA Rubric aligns with the Orbis Education Best Practices for Online Teaching and Course Design and professional standards. The rubric is largely based on Chickering and Gamson’s principles for teaching. The Chickering and Gamson (1987) study of best principles for undergraduate education is the most common practices (or principles) cited for online teaching.  

The overall course design score is based on the percentage of total possible points for course design criteria. Expectations for course design quality are defined as follows:
  1. Meets Criteria: Item meets 85%–100% of criteria as described.
  2. Minor Issues: Item meets 75%–84% of criteria as described.
  3. Major Issues: Item meets 70%–74% of criteria as described.
  4. Does Not Meet Criteria: Item meets 0–69% of criteria as described.
A report summarizing all the data, findings, and recommendations is provided to the partner to use as a tool to determine next steps regarding continuous improvement.

Data Sources

The QA team reviews and analyzes several sources of evaluation and user data to validate findings. Possible data sources include:
  1. Overall QA course design scores by criterion
  2. QA scores by course
  3. NCLEX pass rate data
  4. Pass rates by course
  5. Program exit survey data
  6. Course evaluation student surveys
  7. Editorial review (spelling, grammar, etc.)
  8. Faculty training data (if applicable)
  9. Learning object usage rates (click rates) 
  10. Item analysis for exams
  11. Success coach qualitative feedback
  12. Qualitative instructor feedback provided to the team

Methodology

For details on the methodology for a QA review: 
  1. QA Methodology ABSN
  2. QA Methodology AH (allied health)

Revision Project Plan

When requested, Orbis E-Learning QA offers assistance with a revision plan based upon the recommendations adopted. The revision plan is developed in collaboration with the partner and internal support team for the partner. The revision plan includes:
  1. Roles and responsibilities
  2. Project outline
  3. Timeline
  4. Communication plan
  5. Review plan

Internal Documentation and Tracking

Documentation for course reviews are archived. The QA reports, score sheets, course review notes, review checklists, and revision plans serve as the repository for any pertinent notes and action items. Documentation for course reviews and quality control checks are shared through Box and Smartsheet. 
  1. Preparing for a Partner QA Report Meeting
  2. Example Email to the Partner
  3. QA Revision Project Plan Guidelines
  4. Example Comprehensive or Sample QA  Review Report
  5. Example QA Course Review Notes

    • Related Articles

    • What Is the Timeline for a QA Review?

      What should you expect for a timeline on the various QA deliverables? The timeline for a review depends on the type of review and the number of courses involved. Milestone Checks Each milestone check has its own timeline. Course Design Map. Once an ...

    • What Types of Evaluations Does QA Provide?

      Evaluation Options The Orbis Education e-learning quality assurance (QA) team conducts course-level evaluations throughout the course design, development, implementation, and maintenance life cycle using the QA rubric and score sheet. Evaluation ...

    • Partner Shareable Resources

      Partner Shareable Resources These resources are proprietary but may be shared with the partner. Orbis Education Best Practices for Online Teaching and Course Design Quality Assurance Process. Explains all of the QA evaluation services available to a ...

    • What Is a First-Run Course Review?

      Course evaluation of a live or currently running course is initiated by the internal support team for the partner. These typically involve courses with high fail rates on exams, high attrition rates, or a high number of at-risk reports. An evaluation ...

    • What Is a Student Course Evaluation Dashboard?

      Course evaluation data is another tool available to improve partner programs. Each semester, instructional designers (IDs) gather the course evaluation data from their partners. The ID curates the qualitative data (student comments) while the ...