CharlesLi/llama_2_alpaca_helpful
The CharlesLi/llama_2_alpaca_helpful model is a 7 billion parameter language model, fine-tuned from Meta's Llama-2-7b-chat-hf. This model was trained for 50 steps with a learning rate of 0.0002 and a context length of 4096 tokens. It is a specialized variant of the Llama 2 architecture, intended for helpful conversational tasks.
Loading preview...
Model Overview
CharlesLi/llama_2_alpaca_helpful is a 7 billion parameter language model derived from Meta's Llama-2-7b-chat-hf architecture. This model has undergone a specific fine-tuning process, though the exact dataset used for this fine-tuning is not specified in the provided information.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 0.0002
- Batch Size: 4 (train and eval), with a total effective batch size of 16 due to gradient accumulation
- Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- Scheduler: Cosine learning rate scheduler with a 0.1 warmup ratio
- Training Steps: 50 steps across 2 GPUs
During training, the model achieved a final validation loss of 0.7725. The context length for this model is 4096 tokens.
Intended Use
While specific intended uses and limitations are not detailed, as a fine-tuned variant of a chat-optimized Llama 2 model, it is generally suitable for conversational AI applications where helpful responses are desired.