1 of 25

��Exploring L2 Students’ Writing Performances and Perceptions of Corpus-based Materials

Hui Wang, Anh Dang, Dr. Nina Conrad, Dr. Shelley Staples

University of Arizona

2 of 25

Acknowledgments

Christine Tardy, Shelley Staples, Jeroen Gevers, Course developers
Aleksey Novikov, Emily Palese, Gozde Durgat, Graduate RAs
Kevin Sanchez and Alantis Houpt, Undergraduate RAs
All instructors who participated in this project anonymously

2

3 of 25

Background and Motivation

When it comes to analyzing and responding to sources in academic writing, There are some key features that can specify students’ adoption of research practices:

The use of evaluative adjectives for showing a stance toward an argument (e.g., important, necessary, limited, etc.) (Biber, 2006; Hyland, 2004; Hyland, 2012; Romer, 2010).
The use of modal verbs (e.g. can, could, would, should, etc.) in unreal conditional sentences to express critiques and suggestions to sources (Thompson & Hunson, 2000; Bruce, 2016; Hyland, 2005; Crosthwaite et al., 2017)

Corpus-based pedagogy can effectively guide students to understand language patterns through observing language use in authentic texts across genres (Boulton & Cobb, 2017; Gilmore, 2009; Kennedy & Miceli, 2017).

3

4 of 25

Purpose of the Study

This study explores how L2 students at a U.S. university used the corpus-based materials designed mainly using a learner corpus (Corpus and Repository of Writing) for a Summary and Response project in a L2 writing course.

Research questions:

How were students’ summary and response writing impacted by the corpus-based materials?
What were students’ perceptions of using the corpus-based materials in the summary and response project?

4

5 of 25

Teaching Context

Course information: Undergraduate first-year writing courses designed for L2 writers (English 108) at a southwest U.S. university
Course goals: This course emphasizes the skills of rhetorical analysis, genre analysis, research, reflection, and revision. It helps to prepare students for further research and writing in their future fields of academic work.
Student population: International undergraduate students with various cultural backgrounds and from diverse disciplines
Instructor information: 3 instructors participated in the study during Spring 2022, Summer 2022, and Fall 2022. Among them, 1 instructor had previously used corpora in their research and teaching, while other 2 instructors were novice corpus users and had not used corpora in their teaching before this study.
Modality: in person for Spring and Fall semester, asynchronous online for Summer semester

5

6 of 25

Teaching Intervention

The corpus-based materials used for this study were created using a learner corpus the Corpus and Repository of Writing (Crow) with the focus of evaluative language and unreal conditionals (modal verbs) in summary and response writing.

Crow (the Corpus & Repository of Writing) is a large, online collection of university-level student drafts & instructor materials from First-year Writing Courses at the University of Arizona, Purdue University, & Northern Arizona University.

The corpus-based materials were used as an in-class learning activity between students’ first and final drafts.

6

Corpus-based material handout

7 of 25

Teaching Intervention

The purposes of the corpus-based materials were to develop students’ knowledge, awareness and skills of using evaluative language rather than teaching words or word lists.

Instructors had flexibility to adapt the materials based on their own needs.

Participated instructors encouraged students to use more evaluative adjectives than were in the lists provided. (the participated instructors’ actual teaching materials and classroom observation)

7

8 of 25

Participants

Undergraduate international students enrolled in a first-year writing courses designed for L2 writers (English 108) at a southwest U.S. university
Compared and analyzed students' summary and response drafts from:

Intervention group: 119 students from 6 classes of English 108 participated in corpus-based activities during Spring 2022, Summer 2022 and Fall 2022. Among these students, 24 signed the consent and allowed us to download their drafts for analysis.
Control group: 7 consented students from 2 classes of ENGL 108 in Fall 2022 did not receive corpus-based activities for evaluative language. They also completed summary and response drafts and allowed us to download their drafts for the research.

8

9 of 25

Data Collection

Students’ first and final drafts from both intervention and control groups

24 students’ first and final drafts from the intervention group
7 students’ first and final drafts from the control group

Student survey responses (21 responses) from intervention group

9

10 of 25

Methods

Quantitative analysis: A corpus analysis tool AntConc (version 4.0.11) was applied to investigate the frequency of evaluative adjectives and modal verbs.
Qualitative analysis: manually analyzed stance (positive, neutral, negative) of evaluative adjectives and rhetorical functions of both evaluative adjectives and modal verbs.

10

Evaluative adjectives
Stance

Positive
Neutral
Negative

Rhetorical functions

Agree with the claim (positive)
Highlight significance (positive)
Highlight credibility (positive)
Provide balance and objectivity (neutral)
Express doubt and uncertainty (negative)
Indicate weaknesses and limitations (negative)

Modal verbs
Rhetorical functions
Express doubt and criticism
Give advice and suggestions
Express possibility and potential consequences
Indicate weaknesses and limitations (negative)

11 of 25

The Use of Evaluative Adjectives Listed in the Material

11

12 of 25

The Use of Evaluative Adjectives Listed in the Material

Intervention group: added more evaluative adjectives from the two lists to their final drafts: interesting, useful, helpful, accurate, inconclusive.

First drafts: Normed frequency of the evaluative adjectives provided in the material: 0.72

Final drafts: Normed frequency of the evaluative adjectives provided in the material: 0.78

Example 1: This topic was extremely interesting for me as it demonstrated the contrasts in thinking of two different cultures.

Example 2: This study is very useful, and it can help the universities to understand the problem of international students in this aspect.

Example 3: Senel Poyrazli and Marcos Damian Lopez's article was really helpful for me as an international student and I believe that this would be helpful for those who are intending to come to the U.S

12

Check: Activity Part I, II, and III

13 of 25

The Use of Evaluative Adjectives Listed in the Material

Intervention group: 5 out of 24 students have added one or more evaluative adjectives from the given lists to their final drafts.
Control group: no students made changes regarding these evaluative adjectives between their first and final drafts.
The normed frequency of these evaluative adjectives was higher in the control group. However, the intervention group showed a bit more diversity in their choices.

13

14 of 25

The Use of Evaluative Adjectives in General

The purposes of the activity were to develop students’ knowledge, awareness and skills of using general evaluative language rather than teaching words or word lists.
Based on the corpus-based materials, students were encouraged to brainstorm and use more evaluative adjectives in their drafts.
Intervention group (first vs. final)

Added more evaluative adjectives to their final drafts.

First: occurrences 31, normed frequency 2.23
Final: occurrences 70, normed frequency 3.42

Used more positive and neutral evaluative adjectives in their final drafts.

14

15 of 25

The Use of Evaluative Adjectives in General

Intervention group (first vs. final)

21 out of 24 students made changes regarding evaluative adjectives.
Used more evaluative adjectives highlighting significance of cited sources in their final drafts.

Example: This article makes a great contribution to the life experience of international students abroad and points out which groups are more vulnerable to nostalgia and discrimination.

Used more evaluative adjectives highlighting credibility of cited sources in their final drafts.

Example: 400 surveys sent to U.S. students and 360 samples sent to Internationals is an acceptable number to generalize over the university.

15

16 of 25

The Use of Evaluative Adjectives in General

Control group (first vs. final)

2 out of 7 students made changes regarding evaluative adjectives.
Normed frequency of evaluative adjectives decreased.

First: 2.67 Final: 2.48

Less variety: they mainly used positive evaluative adjectives with the “highlight significance” function.

16

17 of 25

The Use of Evaluative Adjectives in General

Intervention group vs. Control group

First drafts from both groups had similar overall normed frequency of evaluative adjectives. However, the intervention group final drafts contained more evaluative adjectives than the control group final drafts.

17

18 of 25

Use Modal Verbs to Provide Critiques and Suggestions

A part of the corpus-based materials aimed to develop students’ awareness of using modal verbs to provide critiques and suggestions, so we only analyzed how students used modal verbs for providing critiques and suggestions.

Intervention group (first vs. final)

Used greater frequency of modal verbs to provide critiques and suggestions in their final drafts.

Normed frequency of modal verbs in first drafts: 0.86
Normed frequency of modal verbs in final drafts: 5.67

Clear increase in could, should, would.

Example: I think the authors should include examples about what international students should do to make themselves feel better when they feel these negative emotions.

18

19 of 25

Use Modal Verbs to Provide Critiques and Suggestions

Intervention group (first vs. final)

Used more modal verbs in the unreal conditional sentences to directly give advice or suggestions in their final drafts.

Example: The authors should bring up both positive and negative aspects of culture shock instead of assuming culture shock have bad effects and lead to homesickness.

Used more modal verbs in the unreal conditional sentences to express possibility and potential consequences in their final drafts.

Example: If such racial identity is in line with the student body's demographic at the surveyed universities, that may be sufficient to indicate that international students, in general, are more likely to be discriminated against and to feel homesick.

19

20 of 25

Use of Modal Verbs to Provide Critiques and Suggestions

Intervention group:

12 out of 22 students used modal verbs to provide critiques and suggestions in their final drafts.
11 out of 22 students made changes regarding modal verbs for providing critiques and suggestions.

Control group:

2 out of 7 students used modal verbs to provide critiques and suggestions in their final drafts.
1 out of 7 students made changes regarding modal verbs for providing critiques and suggestions.

20

21 of 25

Analysis of Student Survey Responses

21

Research question 2: What are students’ perceptions of using the corpus-based activities in the summary and response project?

21 responses from the intervention group: 14 from Spring, 5 from Summer and 2 from Fall.

The activity design research team has revised the materials in summer based on the feedback from the survey.

Student survey questions	1 Not at all effective	2 Somewhat ineffective	3 Somewhat effective	4 Very effective	Average scores ((1 not at all effective - 4 very effective)
How clear were the goals of the corpus-based materials given to you?	0%	15%	40%	45%	3.3
How effective did you feel the corpus-based materials were for meeting those goals?	0%	20%	40%	40%	3.2
How effective did you feel the corpus-based materials were for helping you write that project?	0%	41%	24%	35%	2.9
How effective did you feel the corpus-based materials were for improving your language and writing skills more generally?	0%	24%	41%	35%	3.1

22 of 25

Analysis of Student Survey Responses

22

Selected student comments and suggestions:

“The corpus, to me, is extremely helpful when learning new expressions, or remind me of interesting/rare ones that I could use.” (word choices)

“Statistics that collect frequent use of academic words or phrases are really useful

because it offers a wide variety of words.” (word choices)

“The data may explain the terms and the language students use. Therefore, this will help in determining the speech.” (learner language use)

“To be more interactive and not just work.” (make it more interactive and engaging)

“Make more clear the intentions, purposes, and goals of introducing the corpus based materials to students.” (explain activity purposes)

“Some students may feel that they need first to understand better why those materials can be helpful to them and learn more about them instead of just having the materials presented to them and told to use them in their next writing assignment.” (explain activity purposes)

23 of 25

Summary and Future Directions

The draft analysis results highlight that the corpus-based materials developed students’ awareness, knowledge as well as skills of using evaluative language in their summary and response writing.

Student survey responses show that the majority of students found the corpus-based materials effective in achieving their learning goals and improving their overall writing skills.

The study highlights the effectiveness of using learner corpora as teaching resources in L2 writing classes.

23

24 of 25

Summary and Future Directions

For the future corpus-based material or activity design, we might consider providing wider range of evaluative adjectives in the materials, and continue to encourage students to brainstorm and incorporate more evaluative adjectives beyond the given lists in their drafts.

To further improve corpus-based activities, it is important to consider promoting greater student participation and engagement by exploring approaches such as gamification and group collaboration.

It is also important to highly address the connection between the corpus-based materials and the writing projects in the future activity design and teaching practices.

24

25 of 25

Thank you!

Questions?

For more information about the project, contact

Hui Wang hwang0524@arizona.edu

Anh Dang anhdang@arizona.edu

Dr. Shelley Staples slstaples@arizona.edu

25

https://crow.corporaproject.org/

https://writecrow.org/