uran1um1/tenorio-1.5b
Tenorio 1.5b is a 1.5 billion parameter large language model developed by uran1um1, fine-tuned from Qwen2.5 1.5b Instruct. This model specializes in Spanish language proficiency, offering improved grammatical correctness, natural language ability, and better formatting in its outputs. It was trained using continued pretraining on Spanish books and instruction tuning with distilled Q&A pairs, making it suitable for Spanish-centric conversational AI applications.
Loading preview...
Tenorio 1.5b: Enhanced Spanish Language Model
Tenorio 1.5b is the second iteration in the Tenorio series of language models, developed by uran1um1. This model is a fine-tuned version of Alibaba's Qwen2.5 1.5b Instruct, specifically engineered to significantly enhance its Spanish language capabilities and output formatting. It maintains a 32768 token context length, leveraging the base model's architecture.
Key Enhancements and Training
The primary focus of Tenorio 1.5b is to deliver superior performance in Spanish. Observed improvements include:
- Better Formatting: Outputs in Spanish frequently utilize Markdown for improved readability and structure.
- Increased Interactivity: The model tends to ask more clarifying questions or seek user input during conversations.
Its training regimen involved a two-stage process:
- Continued Pretraining (CPT): The model was further trained on over 100 public domain books written in Spanish.
- Instruction Tuning: LoRA tuning was applied using 7077 Spanish Q&A pairs, which were distilled from Bonsai 8b through an automated self-prompting script.
Performance Insights
While formal standardized benchmarks were not conducted, a custom mini-benchmark was developed to compare Tenorio 1.5b against its base model (Qwen2.5 1.5b Instruct) on 592 Spanish language questions. A stronger judge model (GPT-OSS 20b) evaluated responses based solely on grammatical correctness and natural language ability. Tenorio 1.5b demonstrated superior performance:
- Tenorio 1.5b Score: 315/592
- Base Model Score: 277/592
This indicates a notable improvement in Spanish language quality over the base model.
Licensing
Tenorio 1.5b inherits the Apache 2.0 license from its base model, Qwen2.5 1.5b Instruct. The specific modifications and fine-tuning applied by uran1um1 are licensed under the MIT license.