The COUP Framework

The COUP Framework is the Open Education Group’s approach to studying the impact of open educational resources (like open textbooks) and open pedagogy in secondary and post-secondary education.

COUP stands for:

Cost
Outcomes
Usage
Perceptions

Cost

Adopting openly licensed resources can impact a range of financial and cost metrics for students and institutions. Proponents of OER frequently claim that using these resources instead of traditional publisher textbooks or digital materials will save students money in educational costs associated with pursuing an undergraduate degree.

The Cost criteria for assessment provide empirical evidence about the magnitude of the financial impacts of OER adoption.

Examples:

Quantifying the cost of previously assigned textbooks as savings for students
Costs associated with the additional purchase of study guides, flashcards, quizzes, etc.

Outcomes

Given the folk wisdom that “you get what you pay for,” some individuals and organizations worry that student learning will necessarily suffer when students use freely available, openly licensed resources instead of $200 textbooks. OER proponents claim that using these resources instead of traditional publisher textbooks or digital materials increases student access to critical learning materials and expands faculty’s academic freedom, consequently improving student learning outcomes.

The Outcomes criteria for assessment provide empirical evidence about the magnitude and direction of the learning impacts of OER adoption.

Examples:

Changes in the percentage of students receiving a C or better (in the course as a whole, or on a particular assignment)
Changes in rates of completion
Changes in drop rates
Changes in enrollment intensity

Usage

The Usage criteria for assessment provide empirical evidence about how students use OER and the degree to which impacts on learning outcomes co-vary with these uses.

Examples:

The frequency with which students access learning materials
Students report accessing learning materials to prepare for class activities

Perceptions

What do faculty and students think about, and feel toward, Open Educational Resources? How do they judge their effectiveness relative to traditional textbooks? Their rigor and coverage? Do they find the formats, structures, and other design features easy to use? Frustrating? What about other stakeholders, like parents or policymakers – what are their thoughts and feelings toward OER?

The Perceptions criteria for assessment provide empirical answers to questions about student valuation of learning materials.

Examples:

The learning materials are visually appealing (e.g., layout, colors).
The writing is engaging/interesting.
The writing is understandable/clear.
The learning materials use examples well to explain the subject matter.

Review of OER assessment studies using the COUP Framework

Howard, V. J., & Whitmore, C. B. (2020). Evaluating student perceptions of open and commercial psychology textbooks. In Frontiers in Education (Vol. 5, p. 139).

Open educational resources confer a variety of advantages to both student and faculty users. For students, the obvious benefit is cost, as most OERs are free to access. OERs also permit students access to materials without restriction from the first day of courses (Seaman and Seaman, 2019), and students can gain access to materials in a variety of formats and across various devices with little restriction (Watson et al., 2017; Ross et al., 2018). These benefits also extend to faculty, who can adapt materials to best suit their unique courses and learning outcomes due to the shared permissions offered with open licenses (Griggs and Jackson, 2017). Despite these potential benefits, skeptics voice concerns about the quality of OER materials. To address these concerns, the impact of OERs and commercial textbooks have been extensively studied using the COUP (Cost, Outcome, Usage, and Perceptions) framework (Bliss et al., 2013).

Studies on cost may explore student savings, often estimating textbook cost savings for one or many courses before and after transitioning from a commercial text to OERs (e.g., Hilton et al., 2014). Cost studies may also explore ancillary benefits of more affordable textbooks, such as decreased course drop rates, greater persistence, and enrollment intensity (e.g., Fischer et al., 2015; Wiley et al., 2016). The latter studies tend to occur at scale, utilizing institutional data from one or many institutions. OER adoptions are estimated to have saved students over a billion dollars (Scholarly Publishing and Academic Resources Coalition (SPARC), 2019), are associated with greater student persistence in courses (reducing course withdrawals; Clinton and Khan, 2019), and students using OERs appear to take more courses both in semesters when using OER and in subsequent semesters compared to commercial textbooks (Fischer et al., 2015), generating increased tuition revenue for institutions where OER is used (Wiley et al., 2016).

Outcomes research focuses on the direct changes in student course performance, such as changes in grade achievement, students’ persistence in education, and enrollment patterns. Notable large-scale studies on OER efficacy include Fischer et al. (2015) and Colvard et al. (2018). Fischer et al. (2015) reported no statistically significant differences in course grades in most courses (60%), improved grades in some courses (33%), and decreased performance in only one course (6.7%) when faculty transitioned from commercial to OER textbooks. Colvard et al. (2018) conducted a similar study, evaluating the impact of OER adoption on student performance for students across the university of Georgia system. Colvard et al. (2018) reported statically significant improvements in final course grades and a reduction in grades of D, F, and W following the transition from commercial to OER textbooks, yet the disproportionate improvements seen in low-income students, non-white students, and part-time students are the most promising results of the study as they demonstrate that OERs may effectively level the playing field for students traditionally underrepresented in higher education. While OER adoption does not universally improve student performance, most studies indicate that learning outcomes either do not change or improve following a transition from commercial to OER textbooks. See Hilton (2016, 2019) for thorough metanalyses on the efficacy of OERs.

Usage research explores how faculty and students engage with OERs, such as evaluating student engagement with OERs and covarying learning outcomes (Gurung, 2017; Clinton, 2018; Cuttler, 2019). Related to studies explore pedagogical techniques made possible by OER (Wiley and Hilton, 2018), such as student generated or remixed OERs (Randall et al., 2013; Azzam et al., 2017; Jhangiani, 2017), and the open sharing of instructional design and pedagogical techniques (Cronin, 2017).

Finally, perception research explores what a variety of stakeholders – including students, faculty, and administrators – think about OERs. Despite advances in technology, the textbook remains a central feature of most courses, and perception research is particularly valuable when understanding the value of textbooks in a course.

Continue reading Evaluating student perceptions of open and commercial psychology textbooks (Optional)