The ICNALE Corpus �(and what it tells us about how Japanese university students speak and write in English)

MARTIN SPIVEY

AKITA INTERNATIONAL UNIVERSITY

Agenda

• Speaker profile
• Introduction to learner corpora
• The ICNALE Corpus
• Current study on lexical bundles and ICNALE data
• Pedagogical implications
• Limitations
• Conclusion
• Discussion/questions

MARTIN SPIVEY NIIGATA JALT 2/2/2025

2

Speaker Profile

• Born in West Yorkshire, England 🡪 Akita, Japan
• MA TESOL from the University of Birmingham
• Two decades of teaching children, teens and adults (mostly eikaiwa)
• Instruct university EAP courses – Academic Reading, TOEIC, Debate, Presentation Methods etc
• Research interests: data-driven learning, learner corpora, and corpus-assisted critical discourse studies
• Hobbies: running, photography, watching Blaublitz Akita

MARTIN SPIVEY NIIGATA JALT 2/2/2025

3

Learner Corpora

• A learner corpus is a ‘systematic collection(s) of authentic, continuous and contextualized language use (spoken or written) by L2 learners stored in electronic format’ (Callies & Paquot, 2015)
• WRITTEN:
• British Academic Written English (BAWE), 6.5m words, L1/L2 Eng, ESP uni papers
• Cambridge Learner Corpus (CLC), 50m, L2 Eng, exam scripts
• International Corpus of Learner English (ICLE) (Granger et al., 2020), 5.5m, L2 Eng (25 language backgrounds), high proficiency essay writing

MARTIN SPIVEY NIIGATA JALT 2/2/2025

4

Learner Corpora

SPOKEN:

• British Academic Spoken English (BASE), 1.5m words, L1/L2 Eng(?), lecture and seminar transcripts 🡪 X limited student output
• MICASE (Simpson et al., 2002), US university setting, 15 types of speech acts in 4 academic disciplines 🡪 X 12% NNES - not all learners
• Louvain International Database of Spoken English Interlanguage (LINDSEI), Gilquin et al. (2010), 1m+, L1/L2 Eng, interviews
• Trinity Lancaster Corpus (TLC) (Gablasova et al., 2019), 4.2m, L1/L2 Eng, spoken examinations of Trinity College London

MARTIN SPIVEY NIIGATA JALT 2/2/2025

5

L1 Japanese EFL Learner Corpora

• NICT-JLE (NICT Japanese Learner English Corpus) – 2m words – SPOKEN – data taken from Standard Speaking Test (oral proficiency interview test), error-tagged, includes native speakers for comparison, available for download: https://alaginrc.nict.go.jp/nict_jle/index_E.html

​

• JEFLL (Japanese EFL Learner Corpus) – 0.7m words of high school student writing – no longer available online? – link dead

​

• SILS Learner Corpus of English – 3.2m words of written texts by students at the School of International Liberal Studies at Waseda University – not publicly available?

MARTIN SPIVEY NIIGATA JALT 2/2/2025

6

Research

• Contrastive Interlanguage Analysis (CIA) (Granger, 1996) – compare L2 production with target language, or different learner varieties (e.g. L1 Japanese English vs L1 Korean English)

– morphology, syntax, lexis, discourse, ‘over/under-use’ of vocabulary, L1 transfer etc

​

• Computer-aided Error Analysis (Dagneaux et al., 1998) – corpus is annotated for errors 🡪 classify errors & analyze frequent ones. Used for research into corrective feedback – experimental study found written CF led to fewer errors in Ss’ writing (Sarré et al., 2021)

​

​

​

​

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

7

Research

• Lexical ‘teddy bears’ (Hasselgren, 1994) – Norwegian SHS/uni advanced Ls & L1 English SHS Ss – Norwegian Ss often used English words & phrases similar to L1 or very frequent

​

• Make collocations (Altenberg & Granger, 2001; Lin & Lin, 2019; Sawaguchi & Mizumoto, 2022) – tend to be over-used by Ls; some evidence of L1 interference in Japanese Ls across proficiency levels (lexical vs delexical make)

​

​

​

​

​

​

​

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

8

ICNALE Corpora

• International Corpus Network of Asian Learners of English (Ishikawa, 2023)

​

• Around 4000 learners from 10 Asian countries/regions (Japan, China, Hong Kong, India, Korea, Pakistan, Philippines, Singapore, Thailand, Taiwan)

​

• 14,000+ speeches, dialogues, picture descriptions, role-plays and essays

​

• Features L1 Eng speakers for comparative purposes

​

• Link: https://language.sakura.ne.jp/icnale/

MARTIN SPIVEY NIIGATA JALT 2/2/2025

9

ICNALE Corpora

MARTIN SPIVEY NIIGATA JALT 2/2/2025

10

Data taken from Ishikawa (2013, 2019, 2023, 2024)

ICNALE - Written Essays Corpus

• Two essay questions:

“Do you agree or disagree with the following statements?

Use reasons and specific details to support your opinion.

(Topic A) It is important for college students to have a part-time job.

(Topic B) Smoking should be completely banned at all the restaurants in the country.”

​

• Test conditions: 20-40 minutes & 200-300 words per essay, computer, NO dictionaries allowed, spell check

Ishikawa (2023)

MARTIN SPIVEY NIIGATA JALT 2/2/2025

11

ICNALE - Written Essays Corpus

MARTIN SPIVEY NIIGATA JALT 2/2/2025

12

Ishikawa (2023)

*CEFR levels are based on participants self-reporting previous proficiency test scores (TOEIC, IELTS etc) as well as scores from a version of the Vocabulary Size Test (Nation & Beglar, 2007) undertaken prior to data collection.

Japanese Participant Proficiency Levels

ICNALE – Spoken Monologues Corpus

• Same two topics as Written Essays – part-time work and smoking in restaurants
• Data collected via automatic telephone recording system
• 10 minutes –

Qs1-4 personal info 🡪 Q5 self-intro (60s) 🡪 Q6 (SPEECH 1) – 20s to prepare – 60s to answer 🡪 Q7 (SPEECH 1 again) – 10s to prepare – 60s to answer 🡪 Q8 (SPEECH 2) 🡪 Q9 (SPEECH 2 again) 🡪Q10 rate your speech

MARTIN SPIVEY NIIGATA JALT 2/2/2025

13

Ishikawa (2023)

ICNALE – Spoken Monologues Corpus

MARTIN SPIVEY NIIGATA JALT 2/2/2025

14

Ishikawa (2023)

*CEFR levels are based on participants self-reporting previous proficiency test scores (TOEIC, IELTS etc) as well as scores from the Vocabulary Size Test (Nation & Beglar, 2007) undertaken prior to data collection.

Japanese Participant Proficiency Levels

Learner Language – the basics

Your turn! Make a list of the top 7 verbs you think appear in each corpus.

MARTIN SPIVEY NIIGATA JALT 2/2/2025

15

WRITTEN ESSAYS

1. be (41,830/M)
2. have (15,951/M)
3. do (9521/M)
4. think (9455/M)
5. smoke (6632/M)
6. work (4423/M)
7. ban (4252/M) (SM 8th 4126/M)

​

​

​

​

​

SPOKEN MONOLOGUES

1. be (43,120/M)
2. have (14,958/M)
3. do (14,690/M)
4. think (10,130/M)
5. agree (9016/M) (WE 10th 3135/M)
6. smoke (7675/M)
7. work (4931/M)

J WE vs J SM keywords

MARTIN SPIVEY NIIGATA JALT 2/2/2025

16

J WRITTEN ESSAYS

1. though (66 - 332/M)
2. section (54- 272/M)
3. guest (52 - 262/M)
4. relation (50 - 252/M)
5. cook (42 - 211/M)
6. collage (40 - 201/M)
7. thus (40 - 201/M)
8. station (38 - 191/M)
9. likely (38 - 191/M)
10. seem (35 -176/M)

​

​

​

​

​

J SPOKEN MONOLOGUES

1. uh (26 - 536/M)
2. uhh (23 - 475/M)
3. am (15 - 309/M)　
4. ok (10 - 206/M)
5. yeah (9 - 186/M)
6. arubaito (8 - 165/M)
7. hello (7 - 144/M)
8. familymart (7 - 144/M)
9. umm (5 - 103/M)
10. interact (5 - 103/M)

​

(adverb – I am agree)

J WE vs ENS WE keywords

MARTIN SPIVEY NIIGATA JALT 2/2/2025

17

J WE

1. lunch (71 - 357/M)
2. guest (52 - 262/M)
3. collage (40 - 201/M)
4. angry (32 - 161/M)
5. cooking (29 - 146/M)
6. yen (27 - 136/M)
7. accord (24 - 121/M)
8. part-time-job (23 - 116/M)
9. english (23 - 116/M)
10. injure (16 - 81/M)

​

​

​

​

​

ENS WE

1. establishment (25 - 258/M)
2. patron (19 - 196/M)
3. successful (18 - 186/M)
4. lucky (17 - 175/M)
5. dollar (16 - 165/M)
6. resume (16 - 165/M)
7. ventilation (15 - 155/M)
8. additional (14 - 144/M)
9. debate (14 - 144/M)
10. related (13 - 134/M)

​

(according to)

J SM vs ENS SM keywords

MARTIN SPIVEY NIIGATA JALT 2/2/2025

18

J SM

1. uhh (23 - 475/M)
2. am (15 - 309/M)
3. experiment (13 - 268/M)
4. self (10 - 206/M)
5. arubaito (8 - 165/M)
6. moreover (7 - 144/M)
7. familymart (7 - 144/M)
8. explain (7 - 144/M)
9. cram (7 - 144/M)
10. mathematics (7 - 144/M)

​

​

​

​

​

ENS SM

1. uhm (143 - 1361/M)
2. ah (79 - 752/M)
3. definitely (64 - 609/M)
4. section (59 - 561/M)
5. away (59 - 561/M)
6. debt (43 - 409/M)
7. financial (36 - 343/M)
8. quite (36 - 343/M)
9. bit (36 - 343/M)
10. extra (35 - 333/M)

​

(+ school)

(L1 interference)

Formulaic language – going beyond the word

A formulaic sequence is ‘...a sequence, continuous or discontinuous, of words or other elements, which is, or appears to be, prefabricated: that is, stored and retrieved whole from memory at the time of use, rather than being subject to generation or analysis by the language grammar.’ (Wray, 2002, p.9)

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

19

Formulaic language – going beyond the word

• Continuous – e.g., important for, smoking should, be completely banned, think it is, There are three reasons, the importance of money (J WE)
• Discontinuous – e.g., not only *** but also 🡪 money, restaurant(s), smoker(s), themselves

it is *** for 🡪 bad, good, important, necessary

​

• Lexical bundles – AKA n-grams – 2/3/4/5/6 words in a continuous sequence

- (‘...do not constitute a complete structural unit...’ (Jeong & Jiang, 2019, p.190)

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

20

Top 10 key 4-grams – J WE & J SM

MARTIN SPIVEY NIIGATA JALT 2/2/2025

21

Top 10 key 4-grams – J WE & ENS WE

MARTIN SPIVEY NIIGATA JALT 2/2/2025

22

Top 10 key 4-grams – J SM & ENS SM

MARTIN SPIVEY NIIGATA JALT 2/2/2025

23

Formulaic language – some conclusions

• Student errors highlighted

– omission of prepositions (agree with this statement/opinion because)

🡪 agree this/the opinion/statement appears 61 times in the J SM corpus

- omission of articles (to work a part-time job)

​

• Many top 10 key 4-grams are prompt-dependent (part-time job is important, think that smoking should etc) so we need to look at a wider selection in order to spot more language trends

​

• Compared to native speakers, Japanese learners more reliant on simple discourse-organizing bundles (I have two reasons, there are three reasons etc.)

​

• able to seems to be difficult for Japanese SPEAKERS to produce – J SM (5/103), J WE (161/810), ENS WE (102/1052), ENS SM (206/1960)

MARTIN SPIVEY NIIGATA JALT 2/2/2025

24

4-grams research study

• Study investigating key differences & similarities between J WE and J SM 4-gram production
• 4-grams extracted and analysed using corpus software Sketch Engine: http://www.sketchengine.eu
• Structural and functional taxonomies of lexical bundles (Biber et al.,1999; Biber et al., 2004)

​

​

• STRUCTURAL

​

​

• FUNCTIONAL

MARTIN SPIVEY NIIGATA JALT 2/2/2025

25

verb phrase

dependent clause

noun phrase and prepositional phrase

stance expressions

discourse organizers

referential expressions

4-grams research study

• STRUCTURAL

​

• VERB PHRASE (are a lot of, banned at all restaurants, college students have a)
• DEPENDENT CLAUSE (by having a part-time, idea that smoking should, with this opinion because)
• NOUN PHRASE & PREPOSITIONAL PHRASE (bad for our health, for college students to, people who want to)

​

• FUNCTIONAL

​

• STANCE EX (agree with this opinion, that it is important, is very bad for)
• DISCOURSE ORGANIZERS (I have two reasons, with this statement because, reasons why I think)
• REFERENTIAL EX (people who do n’t, part-time job is a, college students have a)

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

26

(Provisional) results

MARTIN SPIVEY NIIGATA JALT 2/2/2025

27

JAPANESE WRITTEN ESSAYS

JAPANESE SPOKEN MONOLOGUES

(Provisional) results

• Proportionally, Japanese learners use more verb phrases when speaking

🡪 have a part-time job, I think it is, don’t want to, agree with this opinion

​

• Japanese speakers also rely more on stance expressions

🡪 bad for our health, agree with the statement, I think smoking should, it is important to

​

• A higher proportion of noun/prepositional phrases appear in the writing – a feature of more ‘academic’ language

🡪 people who don’t, a lot of money, most important thing for, on the other hand

MARTIN SPIVEY NIIGATA JALT 2/2/2025

28

(Provisional) results – some clear differences

MARTIN SPIVEY NIIGATA JALT 2/2/2025

29

Standardized frequency 🡪 per million words

Examples

is not good for

• I disagree with this statement because part-time job is not good for university students because if we have a part-time job we can't study... (SM_JPN_PTJ2_108_B1_1.txt)

​

• I agree with the idea because smoking is not good for our health. (SM_JPN_SMK1_136_B1_2.txt)

​

• ...smoke will also cause lung cancer, especially second-hand smoke is not good for other people. (SM_ENS_SMK2_085_XX_1.txt)

MARTIN SPIVEY NIIGATA JALT 2/2/2025

30

Examples

learn a lot of

• from the part time job we can learn a lot of things for example how do you communicate with people around us. (SM_JPN_PTJ2_079_B2_0.txt)

​

• First, college students can learn a lot of things which they don't study at collage through working. (WE_JPN_PTJ0_030_B1_1.txt)

​

• They can learn a lot of skills and – but, of course, that does depend on which part-time job... (SM_ENS_PTJ1_046_XX_2.txt)

MARTIN SPIVEY NIIGATA JALT 2/2/2025

31

Examples

I disagree with this

• I disagree with this statement because I think college students should study firstly. (SM_JPN_PTJ1_041_B1_1.txt)

​

• I disagree with this opinion because I think it is the best of people to stay without constraint. (WE_JPN_SMK0_114_A2_0.txt)

​

• ...being a student at college or university, I disagree with this point because it would be a distraction or disruption... (SM_ENS_PTJ2_098_XX_2.txt)

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

32

Pedagogical implications

• ICNALE corpus data can inform teachers’ syllabus design or lesson plans

(e.g. provide more instruction on article usage with lower levels)

​

• Train your students to use the website! – students can listen to the audio in SM, watch video in SD

​

• Prepare simple & effective activities – error correction, unknown vocabulary check etc

MARTIN SPIVEY NIIGATA JALT 2/2/2025

33

Limitations

• ICNALE focuses on two main topics only (smoking in restaurants & college students doing part-time work). The former is becoming less relevant now most places are becoming smoke-free?

​

• Corpus data are relatively small – 550 Japanese participants, 250k tokens in WE & SM. The latter has only 150 students contributing 4 files each – beware of making too many generalizations!

​

• Test conditions – how would the data differ if it featured spontaneous conversation or discussion between learners? (difficult to collect!)

MARTIN SPIVEY NIIGATA JALT 2/2/2025

34

Conclusion

• ICNALE corpora – gateway into (Japanese) student language – compare with other nationalities/proficiency levels.

​

• Errors - evidence of article/preposition omission but usually not a major comprehension issue? Some instances of L1 vocabulary influence (arubaito, juku) but rare.

​

• Various research studies on ICNALE data - see Ishikawa (2023) for grammar, pragmatics, assessment etc. Also, Sawaguchi (2024) on target lexical bundles for argumentative writing instruction.

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

35

References

Altenberg, B., & Granger, S. (2001). The grammatical and lexical patterning of MAKE in native and non-native student writing. Applied Linguistics, 22(2), 173-195.

​

Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E. (1999). Longman Grammar of Spoken and Written English. Longman.

​

Biber, D., Conrad, S., & Cortes, V. (2004). If you look at…: Lexical bundles in university teaching and textbooks. Applied Linguistics, 25(3), 371-405.

​

Callies, M., & Paquot, M. (2015). Learner corpus research: An interdisciplinary field on the move. International Journal of Learner Corpus Research, 1(1), 1-6.

​

Dagneaux, E., Denness, S., & Granger, S. (1998). Computer-aided error analysis. System, 26(2), 163-174.

​

Gablasova, D., Brezina, V., & McEnery, T. (2019). The Trinity Lancaster Corpus: development, description and application. International Journal of Learner Corpus Research, 5(2), 126-158.

​

​

​

​

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

36

References

​

Gilquin, G., De Cock, S., & Granger, S. (2010). (Eds.) Louvain international database of spoken English interlanguage. Press universitaires de Louvain.

​

Granger, S. (1996). From CA to CIA and back: An integrated approach to computerized bilingual and learner corpora. In K. Aijmer, B. Altenberg, & M. Johansson (Eds.), Languages in contrast: Papers from a symposium on text-based cross-linguistic studies (pp. 37-51). Lund University Press.

​

Granger, S., Dupont, M., Meunier, F., Naets, H., & Paquot, M. (2020). The international corpus of learner English. (Version 3). Presses universitaires de Louvain.

​

Hasselgren, A. (1994). Lexical teddy bears and advanced learners: A study into the ways Norwegian students cope with English vocabulary. International journal of applied linguistics, 4(2), 237-258.

​

Ishikawa, S. (2013). The ICNALE and sophisticated contrastive interlanguage analysis of Asian learners of English. Learner Corpus Studies in Asia and the World, 1, 91-118. https://doi.org/10.24546/81006678

​

Ishikawa, S. (2019). The ICNALE Spoken Dialogue: A new dataset for the study of Asian learners’ performance in L2 English interviews. English Teaching, 74(4), 153-177. https://doi.org/10.15858/engtea.74.4.201912.153

​

​

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

37

References

Ishikawa, S. (2023). The ICNALE guide: An introduction to a learner corpus study on Asian learners’ L2 English. Routledge.

​

Ishikawa, S. (2024). The ICNALE Global Rating Archives: A new assessment dataset for learner corpus studies. Learner Corpus Studies in Asia and the World, 6, 13-38. https://doi.org/10.24546/0100487713

​

Ishikawa, S. (2024). ICNALE: The International Corpus Network of Asian Learners of English. Retrieved January 27, 2025, from https://language.sakura.ne.jp/icnale/

​

Lin, C. H., & Lin, Y. L. (2019). Grammatical and lexical patterning of make in Asian learner writing: A corpus-based study of ICNALE. 3L: Southeast Asian Journal of English Language Studies, 25(3). http://doi.org/10.17576/3L-2019-2503-01

​

NICT Japan Learner English Corpus. (2012). Retrieved January 31, 2025, from https://alaginrc.nict.go.jp/nict_jle/index_E.html

​

Sarré, C., Grosbois, M., & Brudermann, C. (2021). Fostering accuracy in L2 writing: Impact of different types of corrective feedback in an experimental blended learning EFL course. Computer Assisted Language Learning, 34(5-6), 707-729. DOI:10.1080/09588221.2019.1635164

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

38

References

Sawaguchi, R. (2024). Potential of L1 and L2 Corpora to Identify Target Lexical Bundles for Argumentative Essay Writing. Asia Pacific Journal of Corpus Research, 5(1), 1-21.

​

Sawaguchi, R., & Mizumoto, A. (2022). Exploring the use of make + noun collocations by Japanese EFL learners through a bilingual essay corpus. Corpora, 17(SI), 61-77. DOI: 10.3366/cor.2022.0247

​

Simpson, R., Briggs, S., Ovens, J., & Swales, J. (2002). The Michigan corpus of academic spoken English. The Regents of the University of Michigan.

​

Sketch Engine. (2025). Retrieved January 29, 2025, from http://www.sketchengine.eu

​

Wray, A. (2002). Formulaic language and the lexicon. Cambridge University Press.

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

39

Discussion

Any questions or comments?

​

MARTIN SPIVEY NIIGATA JALT 2/2/2025

40

Martin Spivey

Akita International University

mart.spiv@gmail.com

​