Model Overview

This model, llama2_openo1_safe_o1_4o_default_4000_100_full, is a fine-tuned variant of Meta's Llama-2-7b-chat-hf model. It features 7 billion parameters and was developed by CharlesLi. The fine-tuning process involved a single epoch with a learning rate of 2e-05, utilizing a total batch size of 32 across 4 GPUs.

Training Details

During its training, the model achieved a validation loss of 0.5642. Key hyperparameters included an Adam optimizer with betas=(0.9, 0.999) and epsilon=1e-08, and a cosine learning rate scheduler with a 0.1 warmup ratio. The training was conducted using Transformers 4.44.2 and Pytorch 2.4.1+cu121.

Intended Use

As a fine-tuned version of Llama-2-7b-chat-hf, this model is generally suitable for conversational AI applications, leveraging the base model's capabilities. Specific use cases and limitations are not detailed in the provided information, suggesting a broad applicability for tasks typically handled by Llama 2 chat models.