Queen-28/legal-slm-finetuned-Ratu-Chairunisa
The Queen-28/legal-slm-finetuned-Ratu-Chairunisa is a 3.1 billion parameter Qwen2.5-Instruct model developed by Queen-28. It was fine-tuned from unsloth/Qwen2.5-3B-Instruct-bnb-4bit and optimized for training speed using Unsloth and Huggingface's TRL library. This model features a 32768 token context length, making it suitable for applications requiring processing of longer sequences. Its primary differentiation lies in its efficient fine-tuning process, suggesting potential for specialized tasks.
Loading preview...
Model Overview
Queen-28/legal-slm-finetuned-Ratu-Chairunisa is a 3.1 billion parameter language model developed by Queen-28. This model is a fine-tuned variant of the unsloth/Qwen2.5-3B-Instruct-bnb-4bit base model, leveraging the Qwen2 architecture. It is licensed under Apache-2.0.
Key Characteristics
- Architecture: Based on the Qwen2.5-Instruct model family.
- Parameter Count: 3.1 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial context window of 32768 tokens, enabling it to handle extensive inputs and generate coherent long-form outputs.
- Training Efficiency: The model was fine-tuned with a focus on speed, utilizing the Unsloth library and Huggingface's TRL library, resulting in a 2x faster training process compared to standard methods.
Potential Use Cases
This model is suitable for applications where a compact yet capable language model is required, especially in scenarios that benefit from efficient fine-tuning. Its large context window makes it applicable for tasks involving:
- Processing and generating long documents.
- Conversational AI requiring extensive memory.
- Specialized domain applications where custom fine-tuning is beneficial.