Overview

j05hr3d/Llama-3.2-3B-Instruct-C_M_T-DOLLY is a 3.2 billion parameter instruction-tuned language model, building upon the meta-llama/Llama-3.2-3B-Instruct base. This model has been specifically fine-tuned using the TRL (Transformer Reinforcement Learning) framework, employing Supervised Fine-Tuning (SFT) techniques. It supports a substantial context length of 32768 tokens, making it suitable for processing longer inputs and generating coherent, extended responses.

Key Capabilities

• Instruction Following: Designed to accurately interpret and execute user instructions for various text-based tasks.
• Text Generation: Capable of generating human-like text based on prompts, suitable for conversational agents and content creation.
• Extended Context Handling: Benefits from a 32768-token context window, allowing for more detailed and context-aware interactions.

Training Details

The model's fine-tuning process utilized the TRL library, with specific framework versions including TRL 0.27.1, Transformers 4.57.6, Pytorch 2.10.0+cu128, Datasets 4.8.4, and Tokenizers 0.22.2. This setup ensures a robust and modern training environment for enhancing the base Llama model's performance in instruction-tuned scenarios.