CharlesLi/llama_2_sky_safe_o1_4o_reflect_4000_500_full
CharlesLi/llama_2_sky_safe_o1_4o_reflect_4000_500_full is a 7 billion parameter language model fine-tuned from Meta's Llama-2-7b-chat-hf. This model was trained with a focus on achieving a low loss of 0.6698 on its evaluation set, indicating a specialized fine-tuning process. While specific intended uses and limitations require further information, its Llama 2 base suggests general conversational and text generation capabilities. The fine-tuning process involved a learning rate of 2e-05 and a cosine scheduler over one epoch.
Loading preview...
Model Overview
CharlesLi/llama_2_sky_safe_o1_4o_reflect_4000_500_full is a 7 billion parameter language model derived from Meta's Llama-2-7b-chat-hf. This model has undergone a specific fine-tuning process, evidenced by its reported evaluation loss of 0.6698.
Training Details
The fine-tuning procedure utilized the following key hyperparameters:
- Base Model:
meta-llama/Llama-2-7b-chat-hf - Learning Rate: 2e-05
- Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- Scheduler: Cosine learning rate scheduler with a warmup ratio of 0.1
- Epochs: 1
- Total Batch Size: 32 (across 4 GPUs with 2 gradient accumulation steps)
Training results show a progressive reduction in loss, with a final validation loss of 0.6827 at 200 steps. The model was trained using Transformers 4.44.2, Pytorch 2.4.1+cu121, Datasets 3.0.0, and Tokenizers 0.19.1.
Capabilities & Limitations
As a fine-tuned variant of Llama-2-7b-chat-hf, this model likely retains strong general-purpose conversational and text generation abilities. However, specific intended uses, detailed capabilities, and known limitations are not explicitly provided in the current documentation. Users should refer to the base Llama 2 model's documentation for general characteristics and conduct further evaluation for specific applications.