Kuwain 1.5B: An Arabic SLM via Language Injection

Contribution

Introduces a novel language injection method for building Arabic Small Language Models, enabling efficient Arabic specialization while preserving the original model's capabilities.

Abstract

Enhancing existing models with new knowledge is a crucial aspect of AI development. This paper introduces a novel method for integrating a new language into a large language model (LLM). Our approach successfully incorporates a previously unseen target language into an existing LLM without compromising its prior knowledge. We trained a tiny model with 1.5 billion parameters named Kuwain by injecting the Arabic language into a small open-source model mainly trained in English. Our method demonstrates significant improvements in Arabic language performance, with an average 8% improvement across various benchmarks, while retaining the model's existing knowledge with a minimum amount of the original model's data. This offers a cost-effective alternative to training a comprehensive model in both English and Arabic. The results highlight the potential for efficient, targeted language model expansion without extensive retraining or resource-intensive processes.

Cite this paper

@misc{hennara2025kuwain,
  title         = {Kuwain 1.5B: An Arabic SLM via Language Injection},
  author        = {Khalil Hennara and Sara Chrouf and Mohamed Motaism Hamed and Zeina Aldallal and Omar Hadid and Safwan AlModhayan},
  year          = {2025},
  eprint        = {2504.15120},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2504.15120}
}