CharlesLi/llama_2_sky_safe_o1_4o_reflect_4000_1000_full
The CharlesLi/llama_2_sky_safe_o1_4o_reflect_4000_1000_full is a 7 billion parameter Llama-2-7b-chat-hf model fine-tuned by CharlesLi. This model was trained with a learning rate of 2e-05 and a cosine learning rate scheduler over one epoch. While specific differentiators are not detailed, it is based on the Llama 2 architecture, making it suitable for general conversational AI tasks.
Loading preview...
Model Overview
This model, llama_2_sky_safe_o1_4o_reflect_4000_1000_full, is a fine-tuned version of the meta-llama/Llama-2-7b-chat-hf base model, developed by CharlesLi. It features 7 billion parameters and was trained for a single epoch with a learning rate of 2e-05.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 2e-05
- Batch Sizes:
train_batch_sizeof 4,eval_batch_sizeof 4 - Optimizer: Adam with default betas and epsilon
- LR Scheduler: Cosine type with a warmup ratio of 0.1
- Epochs: 1
During training, a validation loss of 0.5639 was observed at step 100, with a final training loss of 0.5360. The training utilized a multi-GPU setup with 4 devices and a gradient accumulation of 2 steps.
Framework Versions
- Transformers: 4.44.2
- Pytorch: 2.4.1+cu121
- Datasets: 3.0.0
- Tokenizers: 0.19.1
Intended Uses
Based on its Llama 2 Chat foundation, this model is likely suitable for general conversational AI, instruction following, and text generation tasks, though specific optimizations or unique capabilities are not detailed in the provided information.