Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
Lucas Bandarkar*, Benjamin Muller, Pritish Yuvraj
Rui Hou, Nayan Singhal, Hongjiang Lv, Bing Liu
Cross-lingual transfer can be as easy as swapping model layers between LLMs!
- Multilingual gap in English-centric LLMs from pretraining imbalance
- Often lack of high quality task data in non-English languages for finetuning
- For instruction finetuning at scale, difficult to combine many desired qualities
- hard to parallelize finetuning across many directions & then combine
Alan Ansell, et al. 2022. Composable sparse fine-tuning for cross-lingual transfer. In Proceedings of the 2022 Annual Meeting of the Association for Computational Linguistics.
Mostafa Elhoushi, et al. 2024. LayerSkip: Enabling early exit inference and self-speculative decoding. Meta Research.
Prateek Yadav, et al. 2023. TIES-merging: Resolving interference when merging models. In 2023 Conference on Neural Information Processing Systems.
Enneng Yang, et al, 2024. Model merging in LLMs, MLLMs, and beyond: Methods, theories, applications and opportunities. ArXiv.
Mitchell Wortsman, et al. 2022.. Model Soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proceedings of the 2022 International Conference on Machine Learning.
- Why is model souping so popular ?
- increase search space of models during training
- boosts robustness & generalization
- cheap, flexible, fully post hoc
- Composable sparse finetuning
- After finetuning “experts”, note which parameters don’t change much & freeze those before retraining
- Then have sparse updates that are less likely to conflict when merged with other “experts”
- Recent interpretability papers say that top few and bottom few transformer layers have most language-specific parameters
- languages share parameters in the middle layers at higher rates
- If you have no in-language data for a particular task/domain, you can finetune an LLM twice (1) on English task-specific instruction data and (2) on generic instruction data in the target language
- Then you can transfer the task capabilities to the target language by merging the model as shown on the left
- the top & bottom transformer layers come from the language-boosted variant, and the middle layers from the task-boosted variant
Parameter-Level Analysis of SFT
- Finetune Llama 3.1 8B (full instruction SFT) on
- English math data
- “Generic” instruction data in a target language (a mix of whatever open instruction data we can find — translation, NER, QA, etc.)
- We find a consistent pattern in which parameters are changing during SFT
- For math expert, only 2nd half of model, both attn parameters and FFN
- For language experts, first & last layers change most, and only attention parameters
- When math expert’s top & bottom layers only are reverted to base model, boost in math capabilities remain → Eureka!
How we can design model merging methods for SFT that boost cross-lingual transfer ?
Visualization of the magnitude of change during SFT for a Japanese expert (left) and a math expert (right)
- Post hoc merge of math expert and language expert by composing different parameters from each
- Many configurations evaluated and a wide range of setups work well:
- 3-5 bottom layers from language expert, 0-2 top layers
- 0-2 lower transition layers, 0-4 upper transition layers
- transition layers & all other parameters are basic averages of both
- On MGSM in Bengali, Telugu, Japanese, and Swahili, this method consistently outperforms:
- individual experts ⇒ combines learned capabilities without interference
- model souping
- SFT on mix of datasets
- Notably, the max or ceiling of this method is much higher!
- Most effective way to do SFT when you have no in-language task domain
- All post hoc & cheap → its a better model souping for multilingual settings
- What other methods can we come up with if we investigate more how LLMs
deal with multilingual text ???