Model Overview

CharlesLi/llama_2_cot_simplest_alpaca_0_full is a 7 billion parameter language model, fine-tuned from the meta-llama/Llama-2-7b-chat-hf base model. This fine-tuning process utilized a specific "generator dataset," suggesting an optimization for text generation or conversational tasks. The model achieved a loss of 1.1711 on its evaluation set during training.

Training Details

The model was trained with a learning rate of 2e-05, a batch size of 4 (total train batch size of 32 across 4 GPUs), and an Adam optimizer. It utilized a cosine learning rate scheduler with a 0.1 warmup ratio over 1 epoch. The training was conducted using Transformers 4.44.2, Pytorch 2.4.1+cu121, Datasets 3.0.0, and Tokenizers 0.19.1.

Key Characteristics

• Base Model: meta-llama/Llama-2-7b-chat-hf
• Parameter Count: 7 billion
• Context Length: 4096 tokens
• Fine-tuning Focus: Generator dataset, implying specialization in text generation or dialogue.

Intended Use Cases

This model is suitable for applications that benefit from a Llama 2-based chat model specifically adapted for generation tasks. Developers should consider its fine-tuning on a generator dataset when evaluating its fit for conversational AI, content creation, or other text generation needs.