j05hr3d/Llama-3.2-3B-Instruct-C_M_T-DOLLY
j05hr3d/Llama-3.2-3B-Instruct-C_M_T-DOLLY is a 3.2 billion parameter instruction-tuned causal language model, fine-tuned from meta-llama/Llama-3.2-3B-Instruct. This model was trained using Supervised Fine-Tuning (SFT) with the TRL framework, offering a context length of 32768 tokens. It is designed for general instruction-following tasks, leveraging its fine-tuned capabilities for conversational AI and text generation.
Loading preview...
Overview
j05hr3d/Llama-3.2-3B-Instruct-C_M_T-DOLLY is a 3.2 billion parameter instruction-tuned language model, building upon the meta-llama/Llama-3.2-3B-Instruct base. This model has been specifically fine-tuned using the TRL (Transformer Reinforcement Learning) framework, employing Supervised Fine-Tuning (SFT) techniques. It supports a substantial context length of 32768 tokens, making it suitable for processing longer inputs and generating coherent, extended responses.
Key Capabilities
- Instruction Following: Designed to accurately interpret and execute user instructions for various text-based tasks.
- Text Generation: Capable of generating human-like text based on prompts, suitable for conversational agents and content creation.
- Extended Context Handling: Benefits from a 32768-token context window, allowing for more detailed and context-aware interactions.
Training Details
The model's fine-tuning process utilized the TRL library, with specific framework versions including TRL 0.27.1, Transformers 4.57.6, Pytorch 2.10.0+cu128, Datasets 4.8.4, and Tokenizers 0.22.2. This setup ensures a robust and modern training environment for enhancing the base Llama model's performance in instruction-tuned scenarios.