CharlesLi/llama_2_sky_safe_o1_llama_3_8B_reflect_1000_500_full
The CharlesLi/llama_2_sky_safe_o1_llama_3_8B_reflect_1000_500_full model is a 7 billion parameter language model fine-tuned from Meta's Llama-2-7b-chat-hf. This model was fine-tuned on a generator dataset, achieving a loss of 0.7148 on the evaluation set. It is intended for tasks related to text generation based on its fine-tuning process. The model utilizes a 4096 token context length.
Loading preview...
Model Overview
This model, llama_2_sky_safe_o1_llama_3_8B_reflect_1000_500_full, is a fine-tuned variant of Meta's Llama-2-7b-chat-hf architecture. It has 7 billion parameters and was specifically trained on a generator dataset, indicating its primary utility in text generation tasks. During its single epoch of training, it achieved an evaluation loss of 0.7148.
Training Details
The model was trained using the following key hyperparameters:
- Base Model: meta-llama/Llama-2-7b-chat-hf
- Learning Rate: 2e-05
- Batch Size: 4 (train), 4 (eval)
- Gradient Accumulation Steps: 2
- Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- LR Scheduler: Cosine type with 0.1 warmup ratio
- Epochs: 1
Intended Uses
Given its fine-tuning on a generator dataset, this model is likely suitable for:
- Text generation
- Conversational AI (inheriting capabilities from Llama-2-7b-chat-hf)
- Tasks requiring creative or coherent text output