Latxa-Llama-3.1-8B: Basque Language Adaptation
Latxa-Llama-3.1-8B is an 8 billion parameter Large Language Model (LLM) developed by the HiTZ Research Center & IXA Research group. It is built upon Meta's Llama-3.1 architecture and has undergone further training on a substantial Basque corpus (4.3 million documents, 4.2 billion tokens) using language adaptation techniques. This specialized training aims to bridge the performance gap for low-resource languages like Basque, where general-purpose LLMs often struggle.
Key Capabilities & Performance
- Basque Language Proficiency: Demonstrates significant performance improvements over the base Llama-3.1 8B on standard Basque benchmarks and in chat conversations.
- Competitive Benchmarking: Achieved 3rd place in a public arena-based evaluation against models like GPT-4o and Claude Sonnet, outperforming other same-sized competitors.
- Instruction Following: Latxa Instruct models are designed to follow instructions and function effectively as chat assistants.
- Evaluated Datasets: Performance validated across multiple Basque-specific datasets including Belebele, X-StoryCloze, EusProficiency, EusReading, EusTrivia, and EusExams.
Use Cases
- Basque Language Applications: Ideal for any application requiring high-quality natural language processing or generation in Basque.
- Research and Development: Promotes the development of LLM-based technology and research for the Basque language.
Limitations
- Language Specificity: Performance is not guaranteed for languages other than Basque.
- Inherited Biases: As it is based on Llama 3.1, it may carry similar biases, risks, and limitations. Users should consult Llama's Ethical Considerations and Limitations.