1 of 31

Open Ed 2021

Improving �Learning Outcomes Through �Open Pedagogy

Diana Daly & �Cheryl (Cuillier) Casey�

2 of 31

We respectfully acknowledge the University of Arizona is �on the land and territories �of Indigenous peoples. �Today, Arizona is home �to 22 federally recognized tribes, with Tucson being home to the O’odham �and the Yaqui.

Land acknowledgement

Map source: Native Land

3 of 31

Cheryl (Cuillier) Casey

Open Education Librarian

Diana Daly

iSchool Assistant Professor

Head of iVoices Media Lab

About us

4 of 31

Session outline

Project background
iVoices Media Lab and Humans R Social Media
Research on learning outcomes
Key resources and lessons learned
Q&A

Shared at https://sched.co/moQQ

Slide deck
Memorandum of understanding for students
Survey questions for students
Open pedagogy resources

5 of 31

iVoices Media Lab

Project duration: Fall 2020 - Spring 2023

Teaching phase: Fall 2020 - Fall 2021

Mission: Student Media Story Production

HRSM and iVoices were funded through a Distinguished Fellowship Award by the Center for University Education and Scholarship for 3 years.

6 of 31

* Moll, L. C., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory into practice, 31(2), 132-141.

iVoices’ vision:

Integrating students’ “funds of knowledge”* into tech classrooms

7 of 31

Process

On creating the open textbook Humans R Social Media

through iVoices Media Lab

8 of 31

Create textbook stub
Design assignments with student media lab workers

TEACHING PHASE*

9 of 31

Create textbook stub
Design assignments with student media lab workers
Training/workshops in media production

Practice Universal Design for Learning

Training/practice in open culture and licensing

Example: iVoices Innovation Pack

Class students create projects
Assess projects

TEACHING PHASE*

*Live Online

DIANA: In response to these media lab assignments, the students of the course compose text-based stories, audio stories, videos, and graphics. We make these projects low-stakes, each worth no more than 10% of the student's grade, with several due per semester. Here you see our general rubric. For accessibility for all students, we teach practicing the principles of Universal Design for Learning, so instructions are in multiple formats such as videos, writing, and live workshops, with ample repetition. We have lots of workshops and help, not only on producing the projects but on licensing and open culture, include guest presentations by Cheryl as our Open Education Librarian. We also offer students shortcuts for integrating Creative Commons content in their work, such as our iVoices Innovation Pack, which is a playlist of short, Creative Commons-licensed tracks made for class reuse by one of our media lab workers. To help the class find other reusable music for audio stories, students were given extra credit for sharing music they found in a class forum, provided they included the Title, Artist, Source and License (or TASL), and provided the license was reusable for the book.

10 of 31

Fall 2020 (113 students)

License all or nothing; open licensing choices; no deidentification options

82 students contributed work under CC BY

72.57% of the class chose to openly license their work via CC BY and ultimately contributed to the collection we chose from for the textbook.

TEACHING PHASE STEP 7: License projects through MOU

DIANA: At the end of the semester, students fill out a Memorandum of Understanding where they choose what to openly license, for use in the textbook and other materials. I've revised our processes around the MOU every semester to better fit students' understandings and offer increased agency for them. Our first semester, Fall 2020, I was eager for textbook content and I emphasized the benefits of open licensing. I believe this approach was ethical enough, but in hindsight it could have been more student-centered. I also offered for them to choose from a menu of Creative Commons licenses. However, everyone in the 75% of the class who chose to openly license chose CC BY, the open license they'd learned the most about from us. About one-quarter of the class chose All Rights Reserved. Our grading team did not know who made these decisions until after Final Grades were input.

11 of 31

Spring 2021 (130 students)

Had options to license all, some, or none �CC BY; could choose full name (36), first name only (18), or be fully deidentified (17)

43 elected to openly license all of their work
27 elected to openly license some of their work

53.85% of the class chose to openly license at least some of their work and ultimately contributed to the collection we can choose �from for the textbook.

TEACHING PHASE STEP 7: License projects through MOU

DIANA: In our 2nd semester of the project, Spring 2021, I was beginning to organize a mountain of content from the previous semester. Knowing we had far more than we needed, I presented the decision around the MOU more in the context of students' lives. I said, “This is your decision. Don't worry about us. We have plenty of content. Contribute and openly license if that truly feels good to you.” Ultimately a little over half of the students chose to openly license and contribute work, a lower percentage than the previous semester, perhaps due to the more student-centered approach.

I no longer offered license options other than CC BY and All Rights Reserved in Spring 21. However, I did offer to partially or fully deidentify their work, and almost half of those who contributed elected this. Offering deidentificaiton has turned out to be very time-consuming for our media lab team, considering it involves video, audio, and graphic editing. But it feels like a student-centered practice and I am continuing it for our final semester, Fall 2021.

12 of 31

Organize by license
Migrate to Google Sheets & Drive
Transcribe & deidentify

COLLECTION PHASE

DIANA: At the end of each teaching phase, the collection phase begins. I supervise a team of interns, usually graduate students in our Library and Information Science masters program, to process our collection of student work. This has taken a few semesters to refine, but the spreadsheet at the top is a visualization of how we do it now. We migrate all of the openly licensed work to Google Drive, and link and contain all we can in a Google spreadsheet. The Google suite offers granular levels of security and useful previews of content from the spreadsheet. Each contributing student is a row and projects and their components are divided and by columns. For Universal Design for Learning we transcribe all audio and video. And when students request it, we remove their last names or full names along with other identifying information about themselves and others in their stories.

13 of 31

Selection of content for textbook

PUBLISH PHASE

Content that is self-reflexive/critical of the norms of social media/society

For example:

talking about toxic masculinity
pushing against gender norms
questioning of dominant modes of labeling and thought
experiences that are underrepresented in academia and media
experiences that are highly represented among students and also interesting, salient, or surprising
Content authored by BIPOC, LGBTQ, or representative of these experiences
Content that really stood out to any reviewer (let's say each reviewer's top 3)

Diversity and Inclusion Guidelines

Created with Mario Villa MLiS, KR18

DIANA: The Publish phase is the next phase. The team of interns and I select content for the textbook. I am still refining these processes, but an overarching theme has been inclusion of underrepresented perspectives, which is what iVoices is about. iVoices amplifies students voices because they offer something different than what we find in scholarship and popular sources, and we particularly value critical perspectives and reflexivity. We also look for representative experiences, for example the most interesting or salient iterations of stories we hear over and over. Students who work with the collection also make recommendations on what stands out to them personally, but on intern teams comprised primarily of white women like myself, it's been essential to gently, constantly decenter hegemonic perspectives.

14 of 31

Selection of content for textbook
Migrate
Organize for export

PUBLISH PHASE

UA CAMPUS REPOSITORY

DIANA: As part of the Publish phase, we also organize the spreadsheet for export to our Campus Repository and other unseen reusers. This means we remove absolutely everything that is not for the intended recipient. Our first repository collection is in development now. We also use student content in our own media, including our website and our podcast Social Media and Ourselves. These stories are not like other open textbook content I've seen or read about. They're sometimes quite personal, as you can hear in this podcast episode about a student who appeared in a Disney commercial talking about her son and a Disney scholarship program she participated in. <PLAY STARTING AT 2:10 UNTIL "THEY DON'T KNOW ME OR MY LIFE".> Kris Kelley was my student and then interned for iVoices working with student stories, and this was the story she chose to tell. She chose the Creative Commons music for it, and she was very happy with what we produced.

15 of 31

Humans R Social Media

https://ivoices.ischool.arizona.edu/humans-r-social-media for the most up-to-date version

Find it at

Open �Textbook Library

MERLOT

Pressbooks Directory

University �of Arizona

16 of 31

Fall 2020 content

in the

Spring 2021 version

17 of 31

Video by Jacquie Kuru. Graphic profile pictures by Trinity Norris (top) and Alyssa de Leon.

All student work shown licensed CC BY.

18 of 31

Students

added new

glossary terms

19 of 31

HRSM is iterative. New �versions will be out periodically.

Goal: 50% of textbook content �will be by students by 2022

20 of 31

Preston Pierce Kersting

Spring 2021 content

in the version we’re

working on now

Reused media TASL: Music includes [Melody 6] from iVoices Innovation Pack by Gabe Stultz, iVoices Media Lab, CC-BY. https://soundcloud.com/socialmediaandourselves/sets/ivoices-innovation-pack-by-gabe-stultz All graphics include icons from The Noun Project, CC-BY. https://thenounproject.com/

21 of 31

"My name is Emilee Gustafson and I'm from Minnesota! I’m currently majoring in studio art and I play wheelchair basketball for the U of A."

DIANA: <don’t play video clip> If you press play on this presentation on your own, you'll hear Emilee Gustafson's story. She never thought she would be an influencer on TikTok, but a TikTok she made referring to wheelchair basketball went viral. She talks about making content to please all these new fans, and then struggling with trolls to the point that she took her content down. I'm a new media scholar, and this kind of story is the essential backstory of what we see and don't see online. I am so grateful to Emilee and all the students who shared their work, and for the next year of the project I'm eager to analyze our collections qualitatively. In the meantime, I'm grateful to Cheryl Casey for studying the learning outcomes of this project from our first two semesters. Cheryl, I'll turn it over to you now.

22 of 31

Research questions

Whether students prefer this type of open pedagogy project �to traditional learning activities (e.g., writing papers, taking quizzes)
How students perceive that participating in this project impacted learning outcomes

Methods

Anonymous survey �of 85 students in �Fall 2020 and 94 students in Spring 2021*
Qualitative & quantitative analysis

* Survey questions were adapted from Hilton, J., Wiley, D., Chaffee, R., Darrow, J., Guilmett, J., Harper, S., & Hilton, B. (2019). Student perceptions of open pedagogy: An exploratory study. Open Praxis, 11(3), 275–288. https://doi.org/10.5944/openpraxis.11.3.973

23 of 31

Student survey results

CHERYL: In all learning outcome categories, students much more frequently rated learning with open pedagogy “better” than “worse.” In Fall, about 66% of students said the HRSM project had greater educational value than traditional learning activities. The following semester, that % increased to 75.5. From Fall to Spring, the % of students who thought learning outcomes were better with open pedagogy increased in all categories except mastering core academic content.

CHERYL: Depending on the question, 2.4%–8.2% found the learning outcomes worse with open pedagogy in Fall. The next semester, those percentages dropped to a range of 0-4.3%.

I attribute the Spring increase in favorable results for open pedagogy to the improvements Diana made in the course after Fall. She clarified instructions, increased the assignment scaffolding, added Pressbooks how-to videos, <and what else, Diana?>

DIANA: In the spirit of UDL we offered instructions for our assignments in more formats, thanks to having more time and media lab workers. And in response to feedback from our Community of Scholars, a board of faculty advising this project, we decreased our reliance on Adobe products and offered students more software options to get to the final product.

CHERYL: Another interesting thing about the data is that in both semesters, the favorable outlook on open pedagogy dropped slightly as students progressed through the survey. One theory is that there was possible question fatigue (students expressed some annoyance in later questions, with comments like “see previous answer”). We could test this theory by mixing up the order of the questions when we survey students this semester.

24 of 31

Imagine a future course you are required to take ... �

If two different sections of this course were offered by the same instructor during equally desirable time slots, but one section had traditional learning activities (such as writing papers and taking tests), and the other used learning activities like participating in the course’s Humans R Social Media project, in which section would you prefer to enroll?

Fall 2020

Spring 2021

11.8% would enroll in the section with traditional learning �activities

62.4% would enroll in the section with activities like the HRSM project

25.8% would �have no preference

35.3% would �have no preference

14.1% would enroll in the section with traditional learning activities

50.6% would enroll in the section with activities like the HRSM project

25 of 31

26 of 31

What students liked about HRSM

“It made us feel like our opinions and our voices were heard and appreciated for the first time.”
“I like how it puts student voices out in the world.”
The opportunity to be published “gave me more incentive to make sure my assignments were my best product.”
It’s “more freeing but equally as challenging.”
“I feel like I learned more and had more motivation �to learn this way.”
“I think it is a great class and that more classes should �be like this.”

27 of 31

Why some students preferred traditional learning activities

“It was very confusing and had some technical issues.”
“Traditional learning activities are easier when you have a massive work load because you know what to expect.”
“I was so worried that I wouldn't have the right way to use all the technology that I felt like I couldn't participate in the learning aspect of it.”
“There were very complex parts within each project and �I seemed to get lost along the way.”
“I learn more by traditional teaching.”�

28 of 31

Key Resources

Knowledgeable Creative Labor

This author/instructor
Paid team of student workers
Unpaid team of graduate student interns
Credited students whose work was integrated into the textbook

Institutional Support

Funding by the institution
Collaboration with librarians in creating licensing agreements
UA’s adoption of Pressbooks, where the book and student work is hosted
Premium support from open education and copyright librarians, Pressbooks, and campus repository
Live Online with Zoom support, & library makerspace for optional workshops

Open Culture Innovations

Resources and practices in open pedagogy
The open source book platform Pressbooks
Creative Commons licensing

Staff coordinator recommended

$ recommended

University OER incentives would have been a great help for recognition and promotion

Continued Pressbooks (or equivalent) development would ease the experience

... and what would have helped even more

Write in research collaboration between instructors and library staff!

DIANA: Lessons I've learned in implementing this project so far relate most to Key Resources. First, Institutional Support has been essential in driving this project, including funding I received from the Center for University Education and Scholarship at our University. I'm especially grateful for Library and technical resources and support. Second, Open Culture innovations including Creative Commons and open pedagogy tools like Pressbooks have made this project possible. Third, Knowledgeable Creative Labor is key to this work, especially that of the students I work with. <ADVANCE TO SEE WHAT WE WISH WE'D HAD>. What we didn't have and what would have helped includes OER incentives at our University, more staff support, and more planned research collaboration. I also look forward to Pressbooks' continued development of areas like the Users section.

29 of 31

With Pressbooks

Highly recommend forming a campus learning community
Highly recommend having another campus partner for support
For large class projects, adding the Single Sign-On option is worth every penny
Pressbooks’ technical support is fantastic!
UA’s “self-service” publishing model limits OER creation capacity
Faculty want more custom �quick guides

With research

Would refine survey questions even further
Grateful that Diana’s grad students helped with analysis (qualitative & quantitative)

Lessons learned on the library side

CHERYL: In terms of supporting and assessing an open pedagogy project from the library side, these are some of my takeaways. I think our Pressbooks learning communities (beginning and advanced) were a valuable way to scale trainings and form a community of practice. I’ve shared all of the resources from our learning communities in the Open Pedagogy Resources handout. With Pressbooks, my library partners with the Digital Learning unit on campus (instructional designers and technologists) on the funding and network management. Adding the Pressbooks Single-Sign On option really reduced emails from students about forgotten passwords and log-in issues. Pressbooks’ technical support is fantastic and I have to give a special shoutout to Amy Song and Steel Wagstaff for their kindness, expertise, and excellent customer service. We operate Pressbooks as a “self-service” model, with no funding for grants, stipends, or course releases. This limits the number of faculty who can afford to participate in OER creation. Developing more custom quick guides (like planning & production checklists) is on my to-do list for the future.

30 of 31

Best practices to keep in mind

Students immersed in knowledge domain
Envision as a collection
Balance sharing encouragement with privacy implications
A new vision of participatory culture
“It was very confusing and had some technical issues.”
“Traditio

CHERYL: There’s much more research to do on this topic, but we think this study and the HRSM textbook show the promise of open pedagogy as a tool for hands-on learning and expanding the diversity of voices in course materials. When creating an open textbook as an open pedagogy practice, plan ahead for student support, autonomy, assessment, and ongoing access. Diana will share some other recommended best practices.

DIANA: First, open pedagogy like this works best in a domain of knowledge in which students are already immersed. Their knowledge will be practical before it's theoretical.

Second, making an open textbook isn't just generating new content to integrate. You are creating a collection. You'll need help organizing and managing that collection. Apply for funding and plan for time for that considerable work, and make sure you include pay for LIS interns.

Third, balance your excitement about this project with deep understanding of the privacy implications of making student work public. Students' heavy use social media in no way translates to an absence of concern about their own privacy.

Fourth, but on a related note, we have the possibility of immersing students in a new vision for participatory culture, where instead of profit at the center there is learning and shared knowledge and transparency.

31 of 31

Questions?�

Resources in Sched�https://sched.co/moQQ

https://ivoices.ischool.arizona.edu/�humans-r-social-media

Diana Daly:�didaly@arizona.edu�

Cheryl (Cuillier) Casey: �ccuillie@arizona.edu