Model Overview

CharlesLi/llama_2_o1_25_full is a fine-tuned version of the Meta Llama 2 7B chat model (meta-llama/Llama-2-7b-chat-hf). This 7 billion parameter model has undergone a single epoch of fine-tuning, resulting in a validation loss of 0.6223.

Training Details

The model was trained using a learning rate of 2e-05, with a total batch size of 32 across 4 GPUs. The optimizer used was Adam with standard betas and epsilon, and a cosine learning rate scheduler with a 0.1 warmup ratio. The training process involved 600 steps, with evaluation loss progressively decreasing.

Key Characteristics

• Base Model: Llama-2-7b-chat-hf
• Parameter Count: 7 billion
• Context Length: 4096 tokens
• Training Loss: Achieved a final validation loss of 0.6223.

Intended Uses

Given its base as a chat model, this fine-tuned version is likely suitable for general conversational AI applications, text generation, and understanding tasks. However, specific intended uses and limitations are not detailed in the provided information.