wvnvwn/llama2-7b-chat-lr5e-5-arcc-lr3e-4-cbasft-new-new
The wvnvwn/llama2-7b-chat-lr5e-5-arcc-lr3e-4-cbasft-new-new is a 7 billion parameter Llama-2 chat model, fine-tuned using AsFT on the ARC-Challenge dataset. This model is specifically optimized for reasoning tasks, leveraging its training on a scientific reasoning benchmark. It is designed for applications requiring enhanced logical inference and problem-solving capabilities within a 4096-token context window.
Loading preview...
Model Overview
This model is a 7 billion parameter Llama-2 chat variant, specifically fine-tuned using the AsFT (Alignment-constrained Supervised Fine-Tuning) method. It builds upon the wvnvwn/llama2-7b-chat-lr5e-5-ssft-cb checkpoint, with further training on the ARC-Challenge dataset.
Key Capabilities
- Enhanced Reasoning: Optimized for tasks requiring logical inference and problem-solving, particularly those found in the ARC-Challenge benchmark.
- Llama-2 Chat Template: Utilizes the standard Llama-2 chat template, ensuring compatibility with existing Llama-2 chat-based applications.
- Merged Checkpoint: The fine-tuned weights are fully merged into the base model, eliminating the need for separate adapters during inference.
Training Details
The model was fine-tuned for 3 epochs on 1,119 examples from the ARC-Challenge training split. Training parameters included a learning rate of 3e-4, a maximum sequence length of 1,024 tokens, and BF16 precision. LoRA configuration used a rank of 16 and alpha of 32, targeting q_proj, k_proj, v_proj, up_proj, and down_proj modules. AsFT employed 160 alignment directions with a lambda_reg of 1.0. Prompt tokens were masked from the training loss to focus on response generation.