1 of 24

“How is it done?” A qualitative analysis of using corpus-based materials to teach L2 writing

Anh Dang, Hui Wang, Dr. Nina Conrad & Dr. Shelley Staples

University of Arizona

2 of 24

Acknowledgments

  • Aleksey Novikov, Emily Palese, Gozde Durgat, Graduate RAs
  • Kevin Sanchez and Alantis Houpt, Undergraduate RAs
  • All instructors who participated in this project anonymously

2

3 of 24

Background and Motivation

  • Data-driven Learning (DDL) has been established as an effective teaching method for second language writing (SLW) courses (Boulton & Cobb, 2017; Charles, 2011, 2014; Chen & Flowerdew, 2018; Cotos, 2014; Flowerdew, 2015; Shin et al., 2018)
  • Many studies testing the effectiveness of DDL materials use a teacher/researcher (or a team of experts) as the creator and deliverer of the materials
  • Many teachers feel they lack training and resources to develop DDL materials, even after some training (Farr, 2008; Schmidt, 2020)

3

4 of 24

Background and Motivation

  • Most previous studies of pre-service and in-service teachers highlight the issues with time to develop materials and difficulties in directly using corpora in the classroom (Heather & Helt, 2012; Lin & Lee, 2015; Bunting, 2013; Breyer, 2009; )
  • Broader impact requires the ability of non-experts to implement DDL materials.
  • Need to investigate how in-service teachers adjust pre-made materials to implement in their classrooms

4

5 of 24

Purpose of the Study

  • This multiple case study aims to explore 4 participating instructors in their process of implementing ready-made corpus-based instructional materials, using a learner corpus (Corpus and Repository of Writing) for a genre-based second language writing course at an US university.

  • RQs:
    • How did the participating instructors use their previous experiences and beliefs about teaching writing with technology to guide the process of implementing the corpus-based materials?
    • How were the participating instructors' perceptions of corpus-based instruction impacted by their participation in the project?

5

6 of 24

What is

Corpus and Repository of Writing (Crow)?

Crow (the Corpus & Repository of Writing) is a large, online collection of university-level student drafts & instructor materials from Foundations Writing Courses at the University of Arizona, Purdue University, & Northern Arizona University.

6

  • The corpus-based materials used for this study were developed using the Crow corpus and feedback from instructors regarding students’ needs.

7 of 24

Context of the Study:

First-year Composition Course for L2 writers

  • First or second course in a sequence of required SLW courses (depending on student’s placement)
  • Course is equivalent to a course for domestic/home students, which has the same assignments and learning outcomes
  • The corpus-based materials were implemented for two (out of four) course assignments:
    • Literacy (Language) Narrative
    • Genre Analysis
  • SLW courses are staffed by instructors with some training in SLW; many instructors are MATESL graduates or in the PhD program in Second Language Acquisition and Teaching

7

8 of 24

Participants: Background

8

Instructor code

How long have you been teaching second language writing?

How long have you been teaching the course that you implemented the corpus-based materials?

Have you taken any classes in which corpus linguistics was discussed?

Have you used corpora in your teaching before participating in this study?

Have you used corpora for other purposes (your own learning, etc.)?

8

Less than a year

Less than a year

Yes

Yes

Yes

9

Less than a year

Less than a year

Yes

Yes

Yes

11

Less than a year

Less than a year

Yes

No

Yes

12

5 years or more

3-4 years

No

No

Yes

9 of 24

Data Collection

  • Phase 1: Focus Groups with Experienced Instructors (a different group of instructors, different from the focal instructors for this study) about the course
    • Focus groups to conduct needs analysis and receive feedback on initial drafts of materials
  • Phase 2: Implementation of the materials with our participating instructors
    • Surveys after each unit of materials implementation (two units with two sets of materials)
    • Observations of their classes: both live online meeting and asynchronous observation of their LMS course.
    • Focus Groups after each unit of implementation
    • Note: Our implementation timeline was during the peak of Covid-19, when all classes were moved online.

9

10 of 24

Corpus-based materials (examples)

10

11 of 24

Design

  • The study is designed as a qualitative study.

    • A qualitative analysis tool Dedoose was first used for manually coding the teaching observations, focus group notes and surveys.

    • We then grouped these codes into different knowledge categories proposed by TPACK framework.

    • Data were coded by two coders. All disagreements were discussed to reach a consensus.

11

12 of 24

Initial Data Analysis

12

13 of 24

TPACK Framework

  • The guiding framework for this study is the TPACK framework.
    • Many studies using TPACK to assess teachers' engagement with corpus-based materials have focused on Technological Knowledge (TK), and to some extent Content Knowledge (“corpus literacy”)
    • We want to use this framework to track the engagement of our participating instructors in all areas, especially Pedagogical Knowledge (PK).

13

Crosthwaite et al., 2021; Farr & O’Keefe, 2019; Ma et al., 2021, 2022; Meunier, 2019; Schmidt, 2022

14 of 24

Our Coding Path

14

15 of 24

Results Overview

15

How students take on certain aspects of the materials

Mode of implementation during Covid time

The format and timeline of the materials

How technology and content can both influence and push against each other

Corpus features were highlighted as affordances

The needs for materials to have more scaffolding, examples and connections to major assignment and the curriculum

Instructors were able to make changes to the materials, provided more scaffolding and made connections to major assignments where they saw fit

Students’ engagement with the materials and their understanding of the purpose of the materials over time

Ready-made corpus-based materials were helpful in the process of implementation

16 of 24

Results Overview

16

The needs for materials to have more scaffolding, examples and connections to major assignment and the curriculum

Instructors were able to make changes to the materials, provided more scaffolding and made connections to major assignments where they saw fit

Students’ engagement with the materials and their understanding of the purpose of the materials over time

Ready-made corpus-based materials were helpful in the process of implementation

17 of 24

Result: Pedagogical Knowledge (PK)

  • Definition: PK describes teachers’ knowledge of the practices, processes, and methods regarding teaching and learning.
    • Directed at the needs for materials to have more scaffolding, examples and connections to major assignment/curriculum.
    • Notes: Sequencing was brought up during the first semester of implementation and changes were made to address that for the second semester of implementation.

17

“Questions can be streamlined a little bit more for “The Genre Analysis as a Genre”. Students have to focus on certain parts and don't seem to be understanding the bigger picture.”

“Maybe we need to provide more scaffolding for these materials since we don’t have that built in. Tense and transitions: students not clear of what to do. More unsure. We need to have more scaffolding.”

18 of 24

Result: Pedagogical Knowledge (PK)

18

“I appreciate that in one of our focus group meetings, the study team explained the rationale behind providing data related to the phrase "for example" in the provided materials. Well, Goal 2 for English 107 focuses on students supporting their ideas with different types of evidence. So it was easy to connect these materials, this data, to my students and their progress, what we had practiced in class, which aligned with target SLOs [student learning outcomes].”

“I think that students don't make the connection between their own writing and what they see in the corpus. Perhaps, making it clearer in the instructions that, for example "This is a transition (for example) that's used often in writing genre analysis, so please take notice and try to apply it, if appropriate, to your own writing".

19 of 24

Result: Pedagogical Content Knowledge (PCK)

  • Definition: PCK describes teachers’ knowledge regarding foundational areas of teaching and learning, including curricula development, student assessment, and reporting results.
    • Instructors’ autonomy: Instructors were able to make changes to the materials, provided more scaffolding and made connections to major assignments where they saw fit.

19

“For the first project, I used the activity on tense to teach synchronously. I asked students: What we use a simple present for? Why we use this tense?; plus questions about grammatically functions. After going through the quiz, I asked in detail about tenses and how to use different tenses.”

I chose to teach the activity “showing vs telling” in synchronous meeting (normally teaches that in in-person classes), with the incorporation of poetry. All the students took quiz together as a class (answering questions then pasting into the quiz format on D2L). I also chose to skip some questions that students can do on their own, since I would normally assign certain things (e.g., write your own scene) as homework.

20 of 24

Result: Technological Pedagogical Knowledge (TPK)

  • Definition: TPK describes teachers’ understanding of how particular technologies can change both the teaching and learning experiences by introducing new pedagogical affordances and constraints.
    • Students’ engagement with the materials and their understanding of the purpose of the materials over time.

20

“First student reflection pulled up: asked students what activity they found most useful, one student said they found all of the activities very helpful because they didn’t understand the conventions of the genre beforehand so they all were really helpful. It’s awesome cause little things are clicking.”

“One reflection the students talked about how they paid more attention to language, especially when looking at Genre Analysis for their paper. A lot of activities are for language use so that was very helpful for the student in the genre that they chose. They did a really good job with the Genre Analysis and that’s probably why.”

21 of 24

Result: Technological Pedagogical Knowledge (TPK)

  • Definition: TPK describes teachers’ understanding of how particular technologies can change both the teaching and learning experiences by introducing new pedagogical affordances and constraints.
    • Ready-made corpus-based materials were helpful in the process of implementation

21

“I think it was easy because it was already embedded into the PDC.”

“It was nice that the materials were provided. For me, the main challenge was to clear out some of the other activities so students wouldn't have too much work for any one due date. ”

“Well, I am mostly using the PDC without many changes

so it was quite smooth for me to incorporate these.”

22 of 24

Implications: Researcher’s perspective

  • Attempting to remove the challenges through adding intro, improving phrasing (with feedback), direction on how to “read” concordance lines and frequency information (in intro video); limited # of concordances and provided variety of activities.
  • Acknowledging the time consuming nature of both doing the research (development of the corpus-based materials) and turning that research into classroom practices.
  • Taking into consideration the instructors’ expertise and experiences when integrating these materials, especially with regard to options to implement these materials into their classroom and a clear timeline about how and when to implement.
  • Having the materials ready to use without the instructors needing to make changes to the materials while keeping a balance with instructor autonomy and agency in allowing them to modify the materials to fit within their classes.

22

23 of 24

Implications: Teacher’s perspective

  • It is important for instructors to get familiar with the corpus-based materials before class and provide sufficient scaffolding to support students during the teaching process.
  • Asking more open-ended questions would be an effective scaffolding strategy to check students’ understanding and evoke reflection on the reasons behind each answer.
  • More structured guidance and support are necessary when instructors take corpus-based activities to a writing class.
  • Integration into pre-designed course so teachers could see example of how to integrate (or use as/is); aligned materials with student learning outcomes

23

24 of 24

Thank you!

Questions?

For more information about the project, please contact

Anh Dang: anhdang@arizona.edu

Hui Wang:hwang0524@arizona.edu

Nina Conrad: ninac@arizona.edu

Shelley Staples: slstaples@arizona.

24