Baked/legal-slm-sft-merged16bit
Baked/legal-slm-sft-merged16bit is a 3.1 billion parameter Qwen2.5-based instruction-tuned causal language model developed by Baked. This model was fine-tuned using Unsloth and Huggingface's TRL library, focusing on specific applications. Its 32K context length supports processing extensive text, making it suitable for tasks requiring deep contextual understanding.
Loading preview...
Model Overview
Baked/legal-slm-sft-merged16bit is a 3.1 billion parameter Qwen2.5-based instruction-tuned language model developed by Baked. It was fine-tuned from unsloth/Qwen2.5-3B-Instruct-bnb-4bit using the Unsloth library and Huggingface's TRL, which enabled a 2x faster training process.
Key Characteristics
- Architecture: Qwen2.5-based, a causal language model.
- Parameters: 3.1 billion, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial 32,768 tokens, allowing for the processing of long documents and complex queries.
- Training Efficiency: Leveraged Unsloth for accelerated fine-tuning.
Potential Use Cases
Given its foundation and fine-tuning methodology, this model is well-suited for applications that benefit from:
- Extended Context: Handling large volumes of text, such as legal documents, research papers, or detailed reports.
- Instruction Following: Responding accurately to specific instructions due to its instruction-tuned nature.
- Resource-Efficient Deployment: Its 3.1B parameter count makes it more accessible for deployment on systems with limited computational resources compared to larger models.