j05hr3d/Llama-3.2-3B-Instruct-C_M_T-AUX_CT_CE_CM-2EP
j05hr3d/Llama-3.2-3B-Instruct-C_M_T-AUX_CT_CE_CM-2EP is a 3.2 billion parameter instruction-tuned causal language model, fine-tuned from Meta's Llama-3.2-3B-Instruct. This model was trained using the TRL framework, focusing on specific instruction-following capabilities. It is designed for general text generation tasks where a smaller, instruction-tuned model is beneficial.
Loading preview...
Overview
This model, j05hr3d/Llama-3.2-3B-Instruct-C_M_T-AUX_CT_CE_CM-2EP, is an instruction-tuned variant of the meta-llama/Llama-3.2-3B-Instruct base model. It features 3.2 billion parameters and a context length of 32768 tokens, making it suitable for tasks requiring moderate context understanding. The fine-tuning process utilized the TRL (Transformer Reinforcement Learning) framework, specifically employing Supervised Fine-Tuning (SFT) to enhance its ability to follow instructions.
Key Capabilities
- Instruction Following: Optimized through SFT with TRL to better adhere to user instructions.
- Text Generation: Capable of generating coherent and contextually relevant text based on prompts.
- Llama 3.2 Architecture: Benefits from the foundational capabilities of the Llama 3.2 series.
Training Details
The model was trained using TRL version 0.27.1, with Transformers 4.57.6, Pytorch 2.10.0+cu128, Datasets 4.8.4, and Tokenizers 0.22.2. The training procedure involved Supervised Fine-Tuning (SFT).
Good For
- Applications requiring a compact, instruction-tuned language model.
- General text generation tasks where the Llama 3.2 architecture is preferred.
- Developers looking for a fine-tuned model with a substantial context window for its size.