Howard, V. J., & Whitmore, C. B. (2020). Evaluating student perceptions of open and commercial psychology textbooks. In Frontiers in Education (Vol. 5, p. 139).

Open educational resources confer a variety of advantages to both student and faculty users. For students, the obvious benefit is cost, as most OERs are free to access. OERs also permit students access to materials without restriction from the first day of courses (Seaman and Seaman, 2019), and students can gain access to materials in a variety of formats and across various devices with little restriction (Watson et al., 2017; Ross et al., 2018). These benefits also extend to faculty, who can adapt materials to best suit their unique courses and learning outcomes due to the shared permissions offered with open licenses (Griggs and Jackson, 2017). Despite these potential benefits, skeptics voice concerns about the quality of OER materials. To address these concerns, the impact of OERs and commercial textbooks have been extensively studied using the COUP (Cost, Outcome, Usage, and Perceptions) framework (Bliss et al., 2013).

Studies on cost may explore student savings, often estimating textbook cost savings for one or many courses before and after transitioning from a commercial text to OERs (e.g., Hilton et al., 2014). Cost studies may also explore ancillary benefits of more affordable textbooks, such as decreased course drop rates, greater persistence, and enrollment intensity (e.g., Fischer et al., 2015; Wiley et al., 2016). The latter studies tend to occur at scale, utilizing institutional data from one or many institutions. OER adoptions are estimated to have saved students over a billion dollars (Scholarly Publishing and Academic Resources Coalition (SPARC), 2019), are associated with greater student persistence in courses (reducing course withdrawals; Clinton and Khan, 2019), and students using OERs appear to take more courses both in semesters when using OER and in subsequent semesters compared to commercial textbooks (Fischer et al., 2015), generating increased tuition revenue for institutions where OER is used (Wiley et al., 2016).

Outcomes research focuses on the direct changes in student course performance, such as changes in grade achievement, students’ persistence in education, and enrollment patterns. Notable large-scale studies on OER efficacy include Fischer et al. (2015) and Colvard et al. (2018). Fischer et al. (2015) reported no statistically significant differences in course grades in most courses (60%), improved grades in some courses (33%), and decreased performance in only one course (6.7%) when faculty transitioned from commercial to OER textbooks. Colvard et al. (2018) conducted a similar study, evaluating the impact of OER adoption on student performance for students across the university of Georgia system. Colvard et al. (2018) reported statically significant improvements in final course grades and a reduction in grades of D, F, and W following the transition from commercial to OER textbooks, yet the disproportionate improvements seen in low-income students, non-white students, and part-time students are the most promising results of the study as they demonstrate that OERs may effectively level the playing field for students traditionally underrepresented in higher education. While OER adoption does not universally improve student performance, most studies indicate that learning outcomes either do not change or improve following a transition from commercial to OER textbooks. See Hilton (2016, 2019) for thorough metanalyses on the efficacy of OERs.

Usage research explores how faculty and students engage with OERs, such as evaluating student engagement with OERs and covarying learning outcomes (Gurung, 2017; Clinton, 2018; Cuttler, 2019). Related to studies explore pedagogical techniques made possible by OER (Wiley and Hilton, 2018), such as student generated or remixed OERs (Randall et al., 2013; Azzam et al., 2017; Jhangiani, 2017), and the open sharing of instructional design and pedagogical techniques (Cronin, 2017).

Finally, perception research explores what a variety of stakeholders – including students, faculty, and administrators – think about OERs. Despite advances in technology, the textbook remains a central feature of most courses, and perception research is particularly valuable when understanding the value of textbooks in a course. A substantial number of studies explore faculty perceptions of OERs, student perceptions of OERs, or both. These studies are vital for a number of reasons. For instance, instructors may rely heavily on the textbook to provide course information, expecting students to come prepared for class discussion (in face-to-face classes). The text may function as an instructor substitute for students in online classes or who do not attend lecture regularly. Faculty may be inclined to select textbooks based on their students’ preferences (Durwin and Sherman, 2008) or their own perceptions of the quality of the material.

Research on student user perceptions of OER tends to favor OER over commercial textbooks. Most students surveyed report that OERs are as good or better than commercial texts in many studies, including those focused within subject matter areas (e.g., health psychology, Cooney, 2017; physics, Hendricks et al., 2017; biology, Watson et al., 2017), in studies across multiple courses at a single institution (e.g., Ikahihifo et al., 2017), and in studies of OER student users across multiple institutions (e.g., Abramovich and McBride, 2018; Griffiths et al., 2018). Many studies also report student preferences for using OER instead of commercial texts. For instance, Delimont et al. (2016) report that students moderately agreed with a statement that they preferred using OER to a commercial textbook (M = 5.7 on a 7-point Likert type scale, 7 indicating strongly agree). Jhangiani and Jhangiani (2017) came at this another way, asking students who had used OER whether they would have preferred to purchase a commercial text; only 20% of respondents indicated slight or strong agreement with this statement; the reader is left to infer that that the other 80% of students preferred not to purchase a text. Similarly, Ross et al. (2018) found that 83% of sociology students surveyed said they would not have preferred to purchase a $100 (CAD) commercial textbook for their course.

Taken together, these studies all indicate strong satisfaction from OER student users. However, specific limitations must be noted. First, these studies have not controlled for familiarity with the course material as a variable influencing choice. That is, students may simply prefer what they already know over other hypothetical course materials. Second, these studies lack any meaningful comparison data – either for comparison materials or comparison to groups using commercial textbooks. Rating the quality of OER materials without providing access to content-matched comparison materials produces only correlational data supporting student preference for OERs. Third, many of these studies do not explore what specific variables were associated with student satisfaction. For instance, research suggests that there may also be an obvious confounding variable in play: rating of the course material as a reflection of student perception of the quality of the course or instructor. For instance, while Griffiths et al. (2018) demonstrated student satisfaction with OER materials, these ratings closely matched student’s ratings of the quality of other aspects of the course, suggesting that student’s perception of the course materials are tied closely with their perception of the quality of the course as a whole. Studies also suggest that there’s a relationship between faculty OER use and students’ perception of faculty. Vojtech and Grissett (2017) found that participants recruited from upper division psychology courses rated a hypothetical faculty member using OER as kinder, more encouraging, and more creative, and a more preferred instructor than a comparative hypothetical faculty member using a commercial textbook. Put simply, multiple course features (such as instructor quality, course design, cost of the text, etc.) may influence student perceptions of OERs beyond the objective quality features of those materials (Hilton, 2016).

Other OER perception studies attempt to control for these limitations by including a commercial textbook-using comparison group. These studies may also evaluate a variety of other use and outcome variables. For instance, Gurung (2017) recruited 1,099 students taking psychology courses at seven institutions and evaluated student satisfaction with the text assigned by their institution, student perception of their class and learning experiences, student performance on a 15-item quiz, and student prior academic achievement (operationalized using ACT standardized college admission test scores). Commercial textbook users reported higher ACT scores, but less satisfaction with learning and with their course. OER users reported that the examples in their texts were more applicable to their everyday life, but reported using fewer study strategies and study aids, and rated their textbook more poorly on elements of visual appeal than students using commercial textbooks. Finally, students using OER performed lower than students using commercial textbooks, even when controlling for ACT scores. Authors speculate that the format of book (i.e., digital OER vs print commercial textbook) may have contributed to these differences.

Jhangiani et al. (2018) went one step further, conducting a quasi-experimental study exploring student perception, textbook use, and course performance in Canadian college students assigned digital OERs, print OERs, or a print commercial textbook. Students were not randomly assigned, and access to the book was a function of the student’s section enrollment. Results indicate that students assigned the OER text (either digital or print) performed similarly or better on course exams than students using the commercial text. There also appeared to be no statistically significant differences in self-reported textbook use. Students rated the print OER textbook higher on seven of 16 quality dimensions adapted from the Textbook Assessment and Usage Scale (TAUS; Gurung and Martin, 2011); there were no dimension on which the commercial text was rated higher than the OER text, and no significant differences between the commercial textbook and the digital OER, suggesting student preference for printed books. Finally, and perhaps most interesting, when students were asked to offer a fair market price for the text they used, students across all textbook types suggested $50 as a fair price for the book.

Cuttler (2019) likewise conducted a quasi-experimental study comparing various student metrics (such as self-reported textbook use, perception of correspondence between assessment and assigned materials, and satisfaction) of students using OERs to students using commercial textbooks, but added the element of comparing students in online and face-to-face sections of four psychology courses. Students were not randomly assigned and access to the book was a function of the students’ section enrollment. Results indicate that students assigned OERs reported using the textbook more (frequency and duration), perceived a greater degree of correspondence between textbook content, class activities, and assessments, and rated the OER textbook as better than the commercial textbook on 11 of 15 TAUS quality dimensions (Gurung and Martin, 2011). These results were generally consistent between students in face-to-face and online sections. Interestingly, 51.68% of students reported that textbooks that they had been required to purchase in the past had not been used enough to justify their cost, with higher reports of underutilization coming from students in face-to-face classes (57%) than online classes (48.2%). Authors speculate that utilization of the textbook may be a function of delivery, with online students relying more on the assigned text due to having fewer opportunities for learning directly from the course instructor. This highlights the valuable role that textbooks play in a course and what instructors likely fear most – that assigned textbooks go unread.

Grissett and Huffman (2019) evaluated the course performance, textbook use, and perceptions of students using OER or commercial textbooks in an introductory psychology course. Unlike Jhangiani et al. (2018) and Cuttler (2019), Grissett and Huffman (2019) were able to hold potential course- and instructor-specific variables constant by comparing sections taught by the same instructor on alternate days. Results indicate no significant difference in student exam performance or textbook use between the two courses. Student satisfaction was evaluated using a 22-question survey adapted from a variety of scales measuring student textbook perception of quality, use, and satisfaction (McGowan et al., 2009; Gurung and Martin, 2011; Gurung and Landrum, 2012; Lindshield and Adhikari, 2013). Unlike previous studies that asked students to evaluate only the textbook they used, Grissett and Huffman (2019) also asked students to rate the potential benefits of a digital textbook versus a traditional printed textbook regardless of the format that the participants used. Results indicate that participants highlighted the biggest advantages of using a digital OER textbook was reduced cost and increased convenience of a portable digital format. Participants indicated the biggest advantages of a printed book were ease of reading, ability to mark up the text, ability to quickly find a topic, and ability to keep the book as a future reference as the biggest advantages of a traditional printed textbook. Finally, when asked to choose one format, most students selected the format of textbook that they were currently using as their most preferred text (69% of traditional/commercial printed textbook users and 70% of free, online textbook users, respectively).

Clinton (2018) controlled for instructor variables by evaluating student performance (assessment score and overall course grades) and perceptions of quality prior to and following the instructors’ transition to an OER textbook in an introductory-level psychology course. Results indicate that students performed slightly better in the semester following OER textbook adoption, though this result is complicated by participants in the OER group reporting higher high school GPAs. There were no significant differences in student use of the textbooks, but there were significantly fewer withdrawals when using the OER textbook, replicating previous findings about the value of OER for promoting student persistence in courses. Last, students perceived the quality of the textbooks to be generally comparable except on the dimension of visual appeal (favoring the commercial text) and the quality of writing (favoring the OER text).

Most of these quasi-experimental studies support the benefit of OER for students, including similar or better performance, reduced costs, and similar or better text quality, as rated by students. The exception to this pattern of more favorable perception for OER textbooks is Lawrence and Lester’s (2018) comparison of student satisfaction and performance prior to and following the instructors’ transition to an OER text. Results indicate that there was no statistically significant difference in performance before and after the transition to OERs, yet fewer students reported satisfaction with the OER (57%) than reported satisfaction with the commercial text in a previous semester (74%). Students also were less likely to endorse satisfaction with various quality elements and the OER text only outperformed the commercial text on ratings of affordability. The authors note that at the time of the study, OER textbook offerings for their field were “relatively immature” (p. 562) and quality ratings could improve in the future as more high-quality materials become available.

The previous studies vary in quality and even quasi-experimental preparations contain critical limitations. These studies have not controlled for student knowledge of the cost of the textbook material as a factor influencing preference. That is, students may demonstrate bias in their reported preferences, rating a free textbook more favorably than a commercial counterpart. To date, the only example of a true experimental preparation evaluating differences in perception of OERs vs commercial texts can be found in Clinton et al. (2019). Researchers recruited and randomly assigned 144 students who had previously taken an introductory-level sociology course to read excerpts from either an OER or commercial textbook. The samples were of approximately equal length, covered related content, and students were not informed of the cost of the course materials nor whether the excerpt was taken from an OER or commercial text. Researchers evaluated student performance on a 10-item learning quiz consisting of five items drawn from each textbook’s test bank. Participants were also asked to complete a 14-item survey adapted from the TAUS (Gurung and Martin, 2011) with open-ended responses to assess perceived quality of the course materials.

Results indicated no significant differences in performance between students assigned the OER or commercial textbooks. Student perception of quality factors were likewise similar except for relevance of photos, better photograph placement, writing engagement, and writing clarity (favoring the commercial text). Students using the OER excerpt rated the sample as better in recency of research findings and using research findings to explain material. Researchers also found positive correlations between student assessment scores and their reported perception of textbook quality, particularly on dimensions related to writing clarity, interest, and the helpfulness and relevance of examples. For students using the commercial textbook, there were significant positive correlations between the learning outcomes and student ratings of textbook interest, textbook helpfulness, relevance of examples, writing engagement, and clarity. For the OER textbook, the only significant correlation between student assessment score and quality was for writing engagement.

Grounded theory analysis of students’ open-ended responses about helpful features of the textbooks indicated that layout features like use of colors, headings, bolded words, and definitions, as well as balance of text and pictures were factors associated with greater usability. It appears that there is a balance to the benefit of adding additional features, as some students referenced visual layout as a distraction, particularly when there was inconsistency in font and an overabundance of callouts in the text. Students also referenced the quality of writing, favoring writing that was clear, simple, and included multiple real-life examples to explain concepts over writing that was too wordy, statistical, “too academic,” or lacked examples.

Taken together, the available literature assesses student satisfaction with OER course materials on self-report surveys. Most quasi-experimental studies demonstrate that students perform as well or better with OER and perceive OER as similar or better in quality than commercial texts (c.f., Lawrence and Lester, 2018). However, quasi-experimental studies cannot control for biases such as student self-selection or uncontrolled external variables between groups, such as instructor differences, changes in course policy, or cohort effects (Hilton et al., 2013). It is also difficult to control for student knowledge of the cost of the textbook unless materials are provided to the students free of charge (e.g., Clinton et al., 2019).