j05hr3d/Llama-3.2-3B-Instruct-C_M_T-SAM_RHO0_02
j05hr3d/Llama-3.2-3B-Instruct-C_M_T-SAM_RHO0_02 is a 3.2 billion parameter instruction-tuned causal language model, fine-tuned from Meta's Llama-3.2-3B-Instruct. This model, developed by j05hr3d, leverages a 32768-token context length and was trained using the TRL framework. It is optimized for general instruction-following tasks, providing enhanced conversational capabilities over its base model.
Loading preview...
Model Overview
This model, j05hr3d/Llama-3.2-3B-Instruct-C_M_T-SAM_RHO0_02, is an instruction-tuned variant of Meta's Llama-3.2-3B-Instruct. It features 3.2 billion parameters and supports a substantial context length of 32768 tokens, making it suitable for processing longer prompts and generating more extensive responses. The fine-tuning process utilized the TRL (Transformer Reinforcement Learning) framework, specifically employing Supervised Fine-Tuning (SFT) to enhance its ability to follow instructions and engage in conversational exchanges.
Key Capabilities
- Instruction Following: Optimized for understanding and executing user instructions effectively.
- Extended Context: Benefits from a 32768-token context window, allowing for more detailed and coherent interactions over longer conversations or documents.
- Base Model Enhancement: Builds upon the robust architecture of Llama-3.2-3B-Instruct, improving its interactive performance.
Training Details
The model was trained using the TRL library (version 0.27.1) with specific versions of Transformers (4.57.6), Pytorch (2.10.0+cu128), Datasets (4.8.4), and Tokenizers (0.22.2). The training procedure involved SFT, focusing on refining the model's responses to various prompts. Further details on the training run are available via Weights & Biases.
Good For
- General-purpose conversational AI applications.
- Tasks requiring adherence to specific instructions.
- Scenarios benefiting from a larger context window for improved coherence and detail.