Developing an internet-based �Tests of Aptitude in Language Learning (TALL): �An open research endeavour
Junlan Pan 盘峻岚
Department of Education
Supervisor: Prof Emma Marsden
Advisor: Dr Giulia Bovolenta
Funder:
Enhancing Research Fund, School of Foreign Languages and Cultures, Chongqing University
Participatory Research Funding, University of York
Background
Individual differences (ID) as variability in explaining L2 learning�
(Brooks et al. 2017; Dᶏbrowska, 2019; DeKeyser, 2012; Dőrnyei, 2005; Kidd et al., 2018; Li et al., 2022; Misyak et al., 2012; Morgan-Short et al., 2014; Wallace, 2020; Yilmaz & Granena, 2019; … )
Language Learning Aptitude as an ID construct
(Abrahamsson & Hyltenstam, 2008; Carroll,1981, 1990; Doughty, 2019; Erlam, 2005; Granena, 2013; Robinson, 1997; Roehr-Brackin, 2021; Saito, 2019; Skehan, 2019; Wen et al., 2019; Winke, 2013; … )
a hybrid construct of cognitive abilities for learning L2 efficiently
General issues
Yes, there are instruments to measure aptitude
e.g. MLAT (Carroll & Sapon, 1959), PLAB (Pimsleur, 1966), LLAMA (Meara, 2005), Hi-LAB (Linck et al., 2013), etc.
However,
Accessibility: LLAMA is openly accessible, but item-level data points are not
concerns about reliability and validity (Bokander & Bylund, 2020; Suzuki, 2021; though see Rogers et al., forthcoming)
Measuring aptitude is crucially important
reducing methodological variations and heterogeneity between studies
reducing the total community work effort and cost of research
facilitating replication and reproducibility in language learning research (Bolibaugh et al., 2021)
conducting high-quality synthesis of aptitude-related research findings
Availability of open instrument
Availability of open data pool
“a chicken-and-egg conundrum”
(Marsden & Morgan-Short, for Language Learning Jubilee)
a sharing platform in L2 research, combining data and measures (MacWhinney, 2017)
“On one hand, before making materials and data openly available, consensus about validity and reliability is desirable; but, critically, in order to reach any consensus, open practices are needed to allow accumulation of knowledge over time and space.”
Develop an internet-based battery with item-level data available for download
Validate this battery with the first batch of datasets
Make the battery an Open Instrument
(in progress)
Make the battery an Open Data Tool
(in progress)
An Open Research Initiative
Tests of Aptitude in Language Learning (TALL)
Theoretical framework informing test design
The Stage Model (Skehan, 2016)
Phonological/Executive (P/E) Model
(Wen, 2016, based on Baddeley & Hitch (1974)
and Engle & Kane (2004))
Materials are available on https://osf.io/czqxt/
as open as possible, as close as necessary
Considerations
Accessibility
Sustainability
not relying on one individual staying in an institutional post (Obels et al., 2020)
Accessibility
√
Interoperability
Reusability
√
√
√
Findability
More work is needed to following the FAIR principles:
www.iris-database.org
More challenges ahead!!
Funding
junlan.pan@york.ac.uk
Time
Thank you!