Xinging/sft_LIMA_template
Xinging/sft_LIMA_template is a 7 billion parameter causal language model, fine-tuned from Meta's Llama-2-7b-hf architecture. It was specifically trained on the LIMA dataset, which focuses on high-quality, diverse instruction-following examples. This model is primarily intended for instruction-following tasks, leveraging its LIMA-based fine-tuning to generate helpful and relevant responses.
Loading preview...
sft_LIMA: Llama-2 Fine-tuned for Instruction Following
Xinging/sft_LIMA_template is a 7 billion parameter language model derived from meta-llama/Llama-2-7b-hf. Its key differentiator lies in its fine-tuning on the LIMA dataset, which is known for its focus on high-quality, human-curated instruction-following examples. This specialized training aims to enhance the model's ability to understand and execute complex instructions effectively.
Key Capabilities
- Instruction Following: Optimized for generating responses that adhere closely to user instructions.
- Llama-2 Base: Benefits from the robust architecture and general language understanding of the Llama-2-7b-hf model.
Good for
- Applications requiring precise and contextually relevant instruction-based responses.
- Developers looking for a Llama-2 variant with enhanced instruction-following capabilities from the LIMA dataset.
Training Details
The model was trained using a learning rate of 2e-05, a total batch size of 32, and a cosine learning rate scheduler with a 0.1 warmup ratio over 3 epochs. The training utilized a multi-GPU setup with 4 devices.