CharlesLi/llama2_openo1_safe_o1_4o_default_4000_100_full
The CharlesLi/llama2_openo1_safe_o1_4o_default_4000_100_full model is a 7 billion parameter language model fine-tuned from Meta's Llama-2-7b-chat-hf. This model was trained for 1 epoch with a learning rate of 2e-05 and achieved a validation loss of 0.5642. It is based on the Llama 2 architecture and is intended for general conversational AI tasks, building upon the safety features of its base model.
Loading preview...
Model Overview
This model, llama2_openo1_safe_o1_4o_default_4000_100_full, is a fine-tuned variant of Meta's Llama-2-7b-chat-hf model. It features 7 billion parameters and was developed by CharlesLi. The fine-tuning process involved a single epoch with a learning rate of 2e-05, utilizing a total batch size of 32 across 4 GPUs.
Training Details
During its training, the model achieved a validation loss of 0.5642. Key hyperparameters included an Adam optimizer with betas=(0.9, 0.999) and epsilon=1e-08, and a cosine learning rate scheduler with a 0.1 warmup ratio. The training was conducted using Transformers 4.44.2 and Pytorch 2.4.1+cu121.
Intended Use
As a fine-tuned version of Llama-2-7b-chat-hf, this model is generally suitable for conversational AI applications, leveraging the base model's capabilities. Specific use cases and limitations are not detailed in the provided information, suggesting a broad applicability for tasks typically handled by Llama 2 chat models.