Introduction 

The Deliverable 2 work group was established in September 2021 to develop base quality course design standards for Colorado Online @. During the fall 2021 semester, the work group dedicated it’s time to reviewing and comparing existing online course design standards, cross-walking the Quality Matters online course design standards to Universal Design for Learning, and identifying the specific gaps—particularly related to diversity, equity, and inclusion—that could be closed by creating new design standards. During the spring 2022 semester, the work group developed checklist tools that would make the new design standards more manageable to interpret and apply to course review and course design processes. The work group presented the course design standards and checklist tools to the Colorado Online @ mini conference in February 2022 and to different faculty/instructor and administrator groups during February 2022 and March 2022, gathering survey feedback from these stakeholder groups. The work group then tested the usability of its core checklist tool (Healthy Course Checklist) in a pilot period that ran from March 2022 to April 2022. The recommendations that follow are the result of this research, prototyping, testing, and feedback process and align with the project vision stated in the Colorado Online @ Consortial Model Overview report (November 2020) to ensure equity of access, support student success, prioritize accessibility, and provide consistent learner experiences within online courses across the system. 

Deliverable 2 Work Group Members 

Amanda Hardman, Learning Designer, Colorado Community College System (CCCOnline) 

Veronica Koehn, Director of Academic Excellence, Northeastern Junior College 

Kelly O’Dell, Psychology Faculty, Community College of Aurora 

Cynthia Krutsinger, Director of eLearning, Pikes Peak Community College 

Kristi Thorland, Instructional Designer, Colorado Community College System (CCCOnline) 

Grace Whiteaker, Senior Learning Designer, Colorado Community College System (CCCOnline) 

Deliverable 2.1. Base Course Design Standards for Colorado Online @ 

The core task of the Deliverable 2 work group was to develop a set of base design standards for online courses delivered within Colorado Online @, including but not limited to Ready-to-Teach (RtT) courses. 

Recommendation 

The subcommittee’s recommendation is to adopt Quality Matters Plus (QM+) standards as the overarching framework for quality online course design offered through Colorado Online @. The QM+ standards are comprehensive and inform implementation of resources and tools provided to faculty/instructors and administrators (including the Healthy Course Checklist recommended in Deliverable 2.2). 

Rationale 

Course design standards are imperative to develop within a consortial model that seeks to ensure a consistent learner experience across colleges. As panelists at an April 2022 WCET summit on “Elements of Quality Digital Learning” advised, course design standards help to make aspirations concrete. Alexandra Pickett, representing SUNY’s OSCQR approach to design standards, pointed out that “if you don’t have a map to get where you want to go, you aren’t going to get there, and any map is better than no map.” With the significance of quality online course design standards in mind, the work group began with a comparison of existing rubrics, including Quality Matters, CSU QLT, SUNY OSCQR, and the California Community Colleges Online Education Initiative Course Design Rubric. The finding was that major quality online course design standards hold key elements in common, including (1) course overviews/introductions and layout, (2) learning objectives, (3) assessment and feedback, (4) learner engagement and interaction, including but not limited to content interaction, (5) course technology, (6) learner support, and (7) accessibility, usability, and universal design.  

Due to the common categories of major online design rubrics, the work group has decided to base our standards on Quality Matters (QM; current higher education rubric represents 6th edition) due to this organization’s depth of research, commitment to continuous improvement, professional development support, and CCCS’s current investment in QM. As of April 2022, CCCS maintains a QM subscription and 6 system colleges have opted to subscribe to QM under the system pricing (ACC, CCD, FRCC, MCC, RRCC, TSC). Other system colleges may not subscribe to QM and yet have adopted internal QA standards based on QM, such as PPCC, who reported using a “light-version of QM” since 2015 in an online course review process survey disseminated by the Deliverable 2 work group. 

In the work group’s review of the 6th edition QM rubric, which currently provides 42 specific course design standards across 8 categories, we identified gaps that were necessary to address to benefit learner success under Colorado Online @, particularly around diversity, equity, and inclusion (DEI). The Colorado Department of Higher Education has identified erasure of equity gaps as a strategic priority, and the CDHE’s Equity Toolkit section on “Teaching & Learning” specifies inclusive course design as a focal point for addressing those equity gaps. We used the Peralta Online Equity Rubric, CAST’S Universal Design for Learning guidelines, and the vast professional development resources, backgrounds, and lived experiences of our work group members to carefully review the existing QM standards and identify the concepts and language needed to incorporate DEI into the current rubric as well as pull out UDL and learner-centered design standards more explicitly. 

In the resultant QM+ standards, we modified the language of 8 existing QM standards and created 24 new “plus” standards. As a single example, Figure 1 below illustrates the insufficiency of QM standard 4.5, “A variety of instructional materials is used in the course.” The work group explored the ambiguity around the word “variety” and unpacked the possibilities into 4 additional QM+ standards that explicitly addressed the value of learner choice of modality, a preference for open educational resources (OER) where possible, the need to include diverse voices that have previously been marginalized in a discipline for an inclusive and equitable learning experience, and the responsibility of a course designer to ensure diverse demographic representation that does not perpetuate harmful stereotypes. While this is only one example, it represents the thoughtful deliberation involved in the intersection between QM course design standards, UDL, and DEI. The full QM+ standards list is available as a Word document download. Appendix A identifies the 24 new QM+ standards compared to their QM counterparts and the conceptual rationale for each addition. 

Figure 1. Quality Matters Plus Example for Standard 4.5 

Note that the QM rubric uses a points-based system since the rubric is part of a course certification process that allows QM subscribers to self-review or opt to participate in a peer review process for QM course certification. This work group has not yet suggested course review processes for Colorado Online @ and does not seek to convey the message that certain standards are nonessential. For instance, the 2020 Consortial Model Overview report identified accessibility as a nonnegotiable, yet the QM rubric only identifies one of its specific accessibility standards as “essential.” 

Factors to Consider 

The following indicates assumptions and other factors to consider when accepting the work group’s recommendation for Deliverable 2.1. 

  • When combining the 42 original QM specific review standards with the new QM+ standards, there is a total of 66 specific standards organized under the 8 original QM categories (called “general standards”). With this heft, it was necessary to condense the QM+ standards for usability. See also Deliverable 2.2. 
  • The number of QM+ standards also necessitates that we consider the amount of time and training it will take to influence college culture as well as increase awareness and receptivity to QM+ standards, particularly those focused on DEI. For instance, the Healthy Course Checklist (HCC) pilot survey found significant challenges in terms of faculty and instructors who felt that DEI concepts did not apply to their disciplines (frequently math, but also economics and technical writing). Some of this response indicated confusion between “teaching” DEI as a topic within the discipline and applying inclusive and equitable strategies as part of one’s pedagogy.  
  • Related, there was feedback from faculty and instructors after both the February 2022 mini conference and the April 2022 HCC pilot that QM alone would be sufficient for identifying quality course design indicators and that QM+ standards, particularly DEI, were not necessary. However, these voices ultimately represented a minority, with most faculty/instructor feedback acknowledging need for and even excited anticipation of the QM+ standards and subsequent access to the HCC tool (see also Deliverable 2.2). 
  • It is important to note that QM updates its rubric every 4-5 years and is scheduled to update in the next year. QM has identified the DEI lens as one of several priorities for the next edition. A few voices in survey feedback did note awareness of this upcoming QM revision. However, our work group does not recommend waiting for the QM rubric revision. One reason is that the Colorado Online @ project timeline is moving forward more quickly than QM’s revision process. Another reason is that we do not know, ultimately, what QM’s decisions will be with respect to DEI and learner-centered design, and there may continue to be gaps as compared to the recommendations our work group has made for the Colorado Online @ project. 
  • Our work group is aware of the need to design regular and substantive interaction (RSI) into online courses per US DOE regulations. Current QM specific review standards 3.5 and 5.3 address RSI with emphasis on planned feedback and learner-instructor interaction. The QM+ modifications to these specific standards are minor but enhance RSI. The DEI and learner-centered design (LCD) deeper-dive checklists (see Deliverable 2.2) can also support RSI, specifically LCD checklist #1, 5, 6, and 8 and DEI checklist #5 and 9. It is still important to recognize, however, that design and facilitation are two separate yet interconnected aspects of the learning experience, and RSI should be considered when developing teaching/facilitation standards. 
  • The work group is working with the assumption that the Colorado Online @ centralized learning design team will build RtT courses according to QM+ base standards. The intention is for QM+ standards to apply to all Colorado Online @ courses through use of the Healthy Course Checklist, but the centralized team only intends to take responsibility for ensuring QM+ standard compliance/accountability for RtT courses (see also Deliverable 2.2). Individual colleges may crosswalk the Healthy Course Checklist to college-specific tools with approval of the Vice Chancelor of Academic and Student Affairs, per the Teaching Excellence and Course Quality Recommendations that will be forthcoming from the Academic Affairs Subcommittee. 

Potential Impact/Effect on Other Decisions 

  • The Learning Design Deliverable 3 (D3) work group is proposing a RtT learning design process that depends on adopting both QM+ standards for RtT courses and the streamlined HCC for all online courses. 
  • The Academic Affairs subcommittee is beginning to develop a faculty/instructor evaluation process that seeks to incorporate the HCC based on QM+ standards. 

Required Resources 

  • Because the QM+ standards are based on the QM rubric, it is necessary to provide ongoing systemwide subscription to Quality Matters (QM) so that all system colleges have access to QM resources. The systemwide QM subscription is already budgeted for the current year to begin May 2022. 

Next Steps 

  • As a next step, the Deliverable 2 work group intends to develop a QM+ guidebook as a necessary resource to support implementation of base standards. A sample page of the proposed guidebook is available at Pressbooks for Standard 1.9+. One of the primary requests received in feedback from the February 2022 mini conference, from stakeholder group meetings, and from the April 2022 HCC pilot was examples (general and discipline-specific), including short embedded video tutorials and a glossary for unfamiliar terms. A guidebook can accomplish much of this work and can complement a scaffolded training program (see also Deliverable 2.2). 

Communication Plan  

  • Between February 2022 and March 2022, the Deliverable 2 work group has presented the QM+ standards and HCC tools to a variety of stakeholder groups, including OFIAC, VPAAs, LTC, SFAC, and the systemwide digital accessibility group. The presentation video for these sessions is available at Quality Matters Plus Overview (7:29). 
  • The QM+ standards will need to be provided to each college to inform use of the Healthy Course Checklist. Currently, the full QM+ standards are available as a link from the Healthy Course Checklist (HCC) document, which is a more condensed and easy-to-interpret version of the QM+ standards for use by faculty, instructors, and supervisors. 
  • Communications will need to use care in distinguishing between QM and QM+. Colleges and individuals who are familiar with the QM rubric and QM peer review process leading to certification will be interested to know how prior QM reviews and certifications will be valued under the new QM+ standards. Ultimately, the QM+ standards ask for explicit attention to DEI and learner-centered elements of design that are not addressed in the QM peer review process or with a QM course certification mark. 

ShapeDeliverable 2.2. Checklists to Support Implementation of Base Standards 

Considering the guidance from the April 2022 WCET summit on “Elements of Quality Digital Learning” that course design standards help to make aspirations concrete, the Deliverable 2 work group was tasked with creating tools that would assist faculty, instructors, learning designers, and administrators with implementing QM+ online course design standards in their own course design and improvement processes. 

Recommendation 

The subcommittee’s recommendation is to adopt a Healthy Course Checklist (HCC) and accompanying, embedded accessibility, DEI (diversity, equity, and inclusion), and learner-centered design checklists for use outside of the centralized Learning Design team. 

Figure 2. Healthy Course Checklist (Image linked to Word document download) 

Rationale 

The QM+ standards for online course design, detailed in Deliverable 2.1, expand the existing QM rubric from 42 to 66 specific standards organized within 8 categories (“general standards”). While this level of itemization may be useful for those with instructional design/learning design backgrounds, this number of standards will easily overwhelm and confuse faculty and instructors who do not bring prior knowledge and experience with learning design into their online learning contexts and yet are often tasked with designing effective online learning experiences for students. For the many instructor-designers or “accidental instructional designers” (Cammy Bean, 2014) in higher education, including within CCCS, it is necessary to condense the QM+ standards into a usable and flexible tool. A tool that is more flexible by design is also important because it is an assumption of the work group that CCCS colleges will ultimately be responsible for integrating the tool into their existing or new course development and course review processes. CCCS colleges will have different ideas about how to review online courses and ensure accountability based on their resources, such as staffing. 

The Deliverable 2 work group developed a core tool, called the Healthy Course Checklist (HCC), that condenses the QM+ standards into a total of 6 categories depicted in Figure 3. The group adopted language of a “healthy course” to promote positive, growth-oriented connotations. Organizing the QM+ standards within 6 criteria communicates clearly the unique focus on an equitable learning experience within an academically sound, or well-aligned, course design. Limiting the criteria to 6 also generates a “QM-lite” experience, or a more manageable cognitive load for those who are not already familiar with the QM rubric nor with principles of instructional/learning design. 

Figure 3. Healthy Course Checklist Criteria 

The full Healthy Course Checklist (HCC) is available as a Word document download. The HCC tool itself does not cite the specific QM+ standards but takes a high-level approach to describing representative strategies that each criterion encompasses, as shown in Figure 4.  

A full-screen version of this image is available for ease of reading: QM+ Healthy Course Checklist Descriptions. 

Figure 4. QM+ Healthy Course Checklist Descriptions 

Although the HCC tool itself is intended to be rather simple in design to present concepts and prioritize strategies through the brief descriptions, the tool is also flexible in design to support deeper engagement for those who are ready to take a more analytic approach to course review and course design. The full set of QM+ standards is linked for reference, and there are currently three deeper-dive checklists embedded within the HCC tool, for criterion 4 (accessibility checklist), criterion 5 (diversity, equity, and inclusion [DEI] checklist), and criterion 6 (learner-centered design [LCD] checklist. The DEI and LCD checklists are aligned back to specific QM+ standards. The direct links to the Word document downloads are included here for convenience: 

The HCC tool promotes a growth mindset through an initially simple set of criteria that embeds resources to “upskill” based on reviewer role within the college’s course review/design processes or based on faculty/instructor professional development growth. The tool also promotes growth mindset through the structure of a two-column worksheet that asks for reflections on (1) ways a HCC criterion is currently met and (2) ways the criterion could be met more thoroughly through course improvement based on the criterion description and any embedded resources consulted (e.g., DEI checklist). The final structural element of the HCC tool is a “Summary for Next Steps” that asks the reviewer (could be self-review or collegial/external review) to identify a few specific priorities along with a timeline and resources needed to act on those priorities. 

While RtT courses are designed through the centralized learning design team and will be carefully designed according to the full set of QM+ standards, the HCC tool represents the minimum standards that individual colleges are asked to adhere to. The HCC tool captures the essence of the full QM+ standards in a simplified version but does not demand that colleges ask their faculty, instructors, and staff to review all online courses at a granular level as we acknowledge the variation that exists at CCCS colleges related to processes and resources such as staffing. The work group’s intent was to create a flexible, growth-oriented tool. 

The Deliverable 2 work group facilitated a pilot of the HCC tool from March 2022 through April 2022 to collect information on the usability of the HCC. The HCC pilot required that participants review an online course design using the HCC, submit the finished HCC, and complete an open-ended survey reporting on their experiences. The pilot ran with a total of 33 completers representing 71% of CCCS colleges (including CCCOnline). The pilot featured strong faculty/instructor voice (79% of pilot completers) and instructor-designer voice (85% of pilot completers designed or participated in the design of the course they reviewed).  

The flexible structure that the work group applied to the HCC design was generally well-received, with several completers expressing appreciation for the simple design and columns to keep track of their notes/observations. A few commenters did express that the broad HCC criteria descriptions could result in vague responses, and several completers indicated that they did not choose to dig deeper into the embedded checklists that would have provided more concrete direction on the criteria. A few of those who did use the embedded checklists still found them to be overwhelming, but whether a completer felt the checklists to be overwhelming or practically helpful also seemed to depend on their prior experiences with accessibility and DEI trainings. Prior knowledge of QM also positively impacted completers’ feedback. In fact, 64% of the completers indicated they had training or experience with QM and this self-identification corresponded to comments about being confused when interpreting the concepts in the HCC tool.  

Ultimately, the HCC pilot reaffirmed ideas the work group already had about next steps needed. Most participants called for examples, discipline-specific examples and tools, exemplar courses, and a variety of trainings including QM, DEI, accessibility, and discipline-specific trainings as well as training in how to complete a course review or look for course evidence. There was common agreement in the HCC pilot survey that the HCC should be a part of a course design or review process at their colleges, with various ideas about how the HCC should manifest in brainstorming sessions, self-reviews, peer reviews, and mentor relationships. There were several comments agreeing that support staff at colleges are needed to support OER, DEI, and accessibility elements of course design/review. and that financial compensation or “credit” may be needed for the respondents to act on the priorities they discovered in their HCC reviews. 

Factors to Consider 

The following indicates assumptions and other factors to consider when accepting the work group’s recommendation for Deliverable 2.2. 

  • The HCC is designed as a growth-mindset tool that is not inherently structured for evaluation. This flexibility is important since individual CCCS colleges may take different approaches to incorporating the tool into existing or new course development and course review processes. Individual CCCS colleges may adapt the structure of the HCC tool as long as the core online course design criteria and links to QM+ standards and accompanying checklist tools are maintained. 
  • The HCC pilot reveals faculty/instructor lack of confidence in applying DEI concepts and web accessibility standards, suggesting that it will take time and training to bring faculty and course designs up to a satisfactory alignment with standards. This is another reason why the HCC tool is designed to encourage growth mindset and continuous improvement over time.  
  • The work group is working with the assumption that the Colorado Online @ centralized learning design team will build RtT courses according to QM+ base standards using all HCC, accessibility, DEI, and LCD checklists. The intention is for core elements of QM+ standards to apply to all Colorado Online @ courses through the HCC tool, but the centralized team only intends to take responsibility for ensuring QM+ standard and HCC compliance/accountability for RtT (see also Deliverable 2.1). The centralized learning design team is available to consult with and assist colleges with fewer resources, if needed. 

Potential Impact / Effect on Other Decisions 

  • The Learning Design Deliverable 3 (D3) work group is proposing a RtT learning design process that depends on adopting both QM+ standards and the HCC. 
  • The Academic Affairs subcommittee is beginning to develop a faculty/instructor evaluation process that seeks to incorporate the HCC based on QM+ standards. 

Required Resources  

  • The HCC tool will need to be available from a centralized location for ease of updating and sharing to system colleges. The checklist tools are currently available as Word document downloads housed on CCCOnline’s media server. Ideally, an HTML version of the checklists with the option to download may better support flexible use, but we also need to determine which resources/checklists we can post on an open-access website such as Pressbooks (example HCC Pressbooks page) is viable considering that the QM foundation of the checklists is considered proprietary information. The work group may require system technology support to store the resources in an accessible yet “fenced” platform. 

Next Steps 

  • Individual CCCS colleges will be responsible for ensuring adherence of their Colorado Online @ courses to the Healthy Course Checklist; CCCS colleges will need guidance to identify quality assurance and accessibility remediation teams as well as guidelines for faculty/instructor accountability. The Deliverable 2 work group will support the Academic Affairs subcommittee in development of these processes.  
  • Many, but not all, CCCS colleges have online course review processes in place. The Learning Design Subcommittee will provide a recommended quality assurance process to CCCS colleges that request this guidance. 
  • The QM+ standards and HCC tool are based in QM; a next step is to develop a QM operational plan that considers college support, staff roles, training, and course review goals.  
  • The need for training has been clear from all stakeholder visits and throughout the HCC pilot. An essential next step is to develop a scaffolded training curriculum, including branching for disciplines and roles. This action item intersects with the next step identified in Deliverable 2.1 to develop a guidebook resource. 
  • Establishing a Colorado Online @ learning design functional group will be a beneficial next step to bring learning/instructional design, eLearning, instructional technology, and similar roles together as a community of practice. This functional group would be poised to collaborate on shared successes, concerns, and solutions as the recommendations from the Learning Design subcommittee are implemented over time.  

Communication Plan 

  • Between February 2022 and March 2022, the Deliverable 2 work group has presented the QM+ standards and HCC tools to a variety of stakeholder groups, including OFIAC, VPAAs, LTC, SFAC, and the systemwide digital accessibility group. The presentation video for these sessions is available at Quality Matters Plus Overview (7:29). 
  • It will be necessary to provide the HCC with its embedded tools to all system colleges. This communication needs to present clear messaging about how the HCC identifies base standard expectations that all Colorado Online @ courses need to be able to meet yet can also be adapted in structure to fit the parameters of specific college-level design and review processes. 
  • Experiences discussing the QM+ standards and HCC tools with individual colleges, including RRCC, CCA, and PPC have suggested it would be beneficial to take a personalized, college-by-college approach to sharing out the HCC tools and discussing its use and alignment with existing or new processes at the specific college. 

ShapeDeliverable 2.3. Centralized Accessibility Support for Colorado Online @ Courses 

The 2020 Consortial Model Overview report identified the need for “[c]entralized instructional designers [who] ensure all courses meet ADA and other accessibility standards,” training on “accessibility strategies,” and “[s]tandard course shells” that are “universally accessible.” Considering both the charges of the consortial model report and the standards emphasizing web accessibility and Universal Design for Learning in Deliverable 2.1, the Deliverable 2 work group determined it would be necessary to make a two-part recommendation for centralized accessibility support. 

Recommendation 

The subcommittee’s recommendation is to provide centralized accessibility support through both a staff position dedicated to accessibility consultation and training as well as an accessibility advisory tool that integrates with the LMS.  

The recommendation can be referred to in the following terms. 

2.3.A. Create a CCCS staff position, a Learning Designer of Accessibility and UDL, within the learning design team to support accessibility of Ready to Teach (RtT) courses and accessibility consultation across the Colorado Online @ colleges. 

2.3.B. Initiate an RFP to acquire an LMS integrated accessibility checker and advisory tool. 

Rationale 

The Deliverable 2.1 recommendation to adopt QM+ standards results in more explicit expectations about web accessibility than the original QM standards. The QM standards that address accessibility are 8.2 and 8.3, which identify the need for text, image, and media accessibility (such as providing closed captioning and audio transcripts). These standards focus on accessible learning materials but not accessible interactions, which is a perspective that the UDL guidelines provide (Guideline 4, Provide Options for Physical Action). Thus, QM+ standards such as 3.4c+ 6.3a+, 6.3b+, and 8.6+ (see Appendix A) broaden the commitment of Colorado Online @ and set a higher level of accessible learning opportunities needed in a “quality” online course. 

As the Deliverable 2 group sought feedback from stakeholder groups during the spring 2022 semester, it became clear that there is a significant conflict between the need to provide accessible learning experiences to benefit diverse learner characteristics and the challenges that exist for faculty, instructors, and administrators who are not confident in their ability to implement accessibility strategies. The March 2022 presentation to the systemwide accessibility stakeholders revealed strong concerns about whether faculty/instructors will be held accountable for closing accessibility gaps in Colorado Online @ course designs. The HCC pilot study ending in April 2022, with 79% of completers representing faculty/instructors, highlighted accessibility as a key concern for fulfilling the HCC criteria. In terms of requested resources, 52% of the HCC pilot participants named accessibility as an area in which they need training and staff support. 

A centralized staff role, a Learning Designer of Accessibility and UDL, will serve as an expert related to web accessibility standards and implementation strategies. This staff person will consult and collaborate with learning design and accessibility service teams at system colleges to help generate solutions to challenges of resource acquisition and distribution as well as accessibility audit and remediation processes. This position will also take a coordinating role for organizing, designing, and delivering ongoing training for the Colorado Online @ consortium related to accessibility and UDL. As a part of the centralized learning design team, this position can provide more direct support to RtT course designs. The Learning Designer of Accessibility and UDL position has been proposed for the FY23 budget by CCCS. 

While a central staff member who can guide training and college consultation is an essential human resource, an LMS accessibility checker and advisory tool is needed to provide on-demand support at the course level across Colorado Online @ courses. Examples of LMS-agnostic accessibility checkers include Blackboard Ally and YuJa Panorama, and the existence of multiple comparable tools means that CCCS will go through an RFP process to identify all possible solutions. Functionality of these tools include course-level checking and reporting, targeted guidance to instructors/users related to how to remediate accessibility issues, and institution-wide reporting on course accessibility and common areas of concern. The data gleaned from an LMS accessibility checker can inform the Learning Designer of Accessibility and UDL’s collaboration with system colleges as well as decisions about training needed. Providing both a human resource and technological resource ensures granular course-level support as well as holistic support to fulfill the vision and intention of Colorado Online @ related to accessible and equitable course design. 

Factors to Consider 

The following indicates assumptions and other factors to consider when accepting the work group’s recommendation for Deliverable 2.3. 

  • The centralized Learning Designer of Accessibility and UDL staff position has been proposed for the FY23 budget by CCCS. 
  • It will be important to acknowledge that adding a centralized staff member with web accessibility and UDL expertise is an appropriate step to focus on training and college consultation but will not be sufficient to fully support processes of accessibility audits and remediation across system colleges. It is ultimately necessary for colleges to prioritize accessible course content and interactions and ensure sufficient staffing and process accountability to fulfill that priority. 
  • The LMS accessibility checker and advisory tool can help with process of remediation at the course level but will not guarantee full accessibility. The faculty/instructor or reviewer is required to act on the guidance provided through the tool, so comprehensive and ongoing training, a culture of commitment to digital accessibility, and processes of accountability are also needed. Paired with thoughtful training efforts and support staff at colleges, the two recommendations taken together can have significant positive impact on accessible learning experiences through Colorado Online @. 

Potential Impact/Effect on Other Decisions 

  • The Learning Design Deliverable 3 (D3) work group is proposing a RtT learning design process that depends on adopting both QM+ standards and the HCC to ensure fully accessible RtT courses. The proposed RtT learning design process assumes that faculty/instructors are able to modify course content, and once modifications occur, the centralized learning design team can no longer assure accessibility. This recommendation supports the commitment to course accessibility once the RtT courses are adopted by individual faculty/instructors. 
  • This recommendation supports the Technology subcommittee’s proposal to acquire the accessible and universally designed ReadSpeaker technology as well as the Technology subcommittee’s forthcoming conversations that prioritize vendor/digital integration accessibility. 

Next Steps 

  • It will be necessary to leverage the systemwide accessibility advisory committee in collaboration with the Technology subcommittee to develop a procurement policy and process to ensure web accessibility of and compatibility with assistive technologies for any tool or technology integrated into D2L. 

Required Resources 

  • The staff role of Learning Designer for Accessibility and UDL has been proposed for the FY23 budget by CCCS. 
  • Pursuing an RFP for an LMS accessibility checker and advisory tool is contingent on budget approval. 

Communication Plan 

  • Between February 2022 and April 2022, the Deliverable 2 work group discussed accessibility and UDL needs with a variety of stakeholder groups, including the systemwide disability directors group and the digital accessibility group. 
  • Adding a centralized staff role and acquiring LMS accessibility checker technology have benefits to supporting faculty/instructor and staff efforts to ensure digital accessibility of online courses, but there are also limitations related to individual college cultures and internal resources. In communicating the centralized resources and training available, it will also be necessary to acknowledge both the affordances and limitations and emphasize college responsibility related to online course remediation and faculty/instructor accountability. 

Appendix A. QM+ Standards Compared to QM Counterparts 

QM Standard QM+ Standard Rationale 
1.4. Course and institutional policies with which the learner is expected to comply are clearly stated within the course, or a link to current policies is provided. 1.4a+. An institutional diversity, equity, and inclusion statement is provided in the course. Inclusion and equity 
1.4 1.4b+. Course policies are written in inclusive, student-centered language, such as using a welcoming or invitational tone in a late policy. Inclusion and equity 
1.9. Learners are asked to introduce themselves to the class. 1.9+. Learners are provided with multiple opportunities to self-identify preferred names and pronouns and use them throughout the course. Inclusion and equity 
2.5. The learning objectives or competencies are suited to the level of the course. 2.5+. The design of the course and module objectives allows learners to engage at multiple cognitive levels. Universal Design for Learning (particularly 8.2) 
3.2. The course grading policy is stated clearly at the beginning of the course. 3.2+. The course grade book is organized logically and aligned to the course grading policy. Clarification of Quality Matters 
3.3. Specific and descriptive criteria are provided for the evaluation of learners’ work, and their connection to the course grading policy is clearly explained. 3.3+. Assessment tools such as rubrics and feedback utilize “not yet” language to encourage a growth mindset. Inclusion and equity, Universal Design for Learning (particularly 8.4) 
3.4. The assessments used are sequenced, varied, and suited to the level of the course. 3.4a+. Assessments are suited to the level of the course to ensure equity. Restructure of Quality Matters 3.4, inclusion and equity 
3.4 3.4b+. Assignments are scaffolded where possible, breaking larger assignments into smaller deliverables and providing resources such as templates, examples, and rubrics. Universal Design for Learning (particularly 5.3) 
3.4 3.4c+. Where possible, choice is built into assessment options (authentic assessment where possible), and all assessment options are web accessible. Universal Design for Learning (particularly 5.1 and 5.2), inclusion and equity 
3.5a+. The course provides learners with multiple opportunities to track their learning progress with timely, learner-centered feedback. (Bold text identifies this as a modification to current QM standard.) 3.5b+. The course exhibits strategies that allow learners to track their own progress, such as providing task lists or checklists. Universal Design for Learning (particularly 6.1, 6.2, 6.3, and 6.4) 
4.5. A variety of instructional materials is used in the course. 4.5a+. The variety of instructional materials provided in each module allows some degree of student choice, such as meeting learning objectives through text versus aural or visual materials. Universal Design for Learning (particularly 7.1, 7.2, and 2.5) 
4.5 4.5b+. Instructional materials incorporate open educational resources (OER) where possible. Universal Design for Learning (particularly 7.2), inclusion and equity 
4.5 4.5c+. Instructional materials represent a diversity of perspectives, including historically marginalized voices. Inclusion and equity 
4.5 4.5d+. Instructional materials exhibit diverse demographic representation that disrupts harmful stereotypes and biases. Inclusion and equity 
5.2a+. Learning activities provide opportunities for interaction that support active learning and meaningful collaborative work. (Bold text identifies this as a modification to current QM standard.) 5.2b+. Learning activities provide space for metacognition, such as incorporating reflection. Universal Design for Learning (particularly 3.4, 6.4, and 7.2) inclusion and equity 
5.2 5.2c+. The variety of learning activities provided in each module allows some degree of student choice, such as completing a practice quiz, practice worksheet, or guided reading. Universal Design for Learning (particularly 7.1 and 7.2), inclusion and equity 
5.2 5.2d+. Learning activities are structured to make connections to prior learning to strengthen new knowledge being built and applied. Universal Design for Learning (particularly 3.1, 3.2, and 3.4), inclusion and equity 
5.4a+. The requirements for reasonable and productive learner interaction are clearly stated. (Bold text identifies this as a modification to current QM standard.) 5.4b+. Activity prompts and course policies or expectations invite learners to contribute to building a shared body of knowledge in the course. Universal Design for Learning (particularly 7.1, 7.3, and 8.3), inclusion and equity 
6.2. Course tools promote learner engagement and active learning. 6.2+. The course provides tools for open communication, such as general discussion forums or video conferencing. Inclusion and equity 
6.3. A variety of technology is used in the course. 6.3a+. Thoughtful and deliberate use of external tools and resources minimizes the need to navigate outside the LMS, protects learner data privacy, and ensures usability across mobile devices and user ability. Universal Design for Learning (particularly 7.3, 5.1, and 5.2), inclusion and equity  
6.3 6.3b+. Alternatives are provided for required activities that rely on tools and technologies outside the learning management system. Universal Design for Learning (particularly 4.1, 4.2, and 7.3), inclusion and equity 
8.1a+. Course navigation facilitates ease of use, including from mobile devices. (Bold text identifies this as a modification to current QM standard.) 8.1b+. The course represents a logical and consistent module organization structure that is as flat as possible, avoiding nested modules. Universal Design for Learning (particularly 3.3 and 7.3), inclusion and equity 
8.4. The course provides alternative means of access to multimedia content in formats that meet the needs of diverse learners. 8.4+. The course design ensures accessibility of learning objects and interactive activities. Universal Design for Learning (particularly 1.1, 4.1, and 4.2), inclusion and equity 
8.6. Vendor accessibility statements are provided for all technologies required in the course. 8.6+. Course design ensures that tools and technologies selected for use are web accessible. Inclusion and equity