CharlesLi/llama_2_cot_simplest_alpaca_2_full
The CharlesLi/llama_2_cot_simplest_alpaca_2_full model is a 7 billion parameter language model fine-tuned from Meta's Llama-2-7b-chat-hf. This model is specifically adapted for conversational tasks, leveraging its Llama 2 base architecture. It was trained with a focus on generating coherent and contextually relevant responses, making it suitable for general-purpose chat applications.
Loading preview...
Overview
This model, llama_2_cot_simplest_alpaca_2_full, is a fine-tuned variant of Meta's Llama-2-7b-chat-hf, a 7 billion parameter causal language model. It has been adapted from the original Llama 2 architecture, which is known for its strong performance in various natural language understanding and generation tasks. The fine-tuning process involved training on a specific "generator dataset" over a single epoch, achieving a reported loss of 0.9462 on the evaluation set.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 2e-05
- Batch Size: 4 (train and eval)
- Gradient Accumulation Steps: 2 (resulting in a total train batch size of 32)
- Optimizer: Adam with default betas and epsilon
- LR Scheduler: Cosine with a 0.1 warmup ratio
- Epochs: 1
Intended Use Cases
Given its foundation on Llama-2-7b-chat-hf and the fine-tuning process, this model is primarily intended for conversational AI applications. It can be used for generating human-like text in response to prompts, engaging in dialogue, and other tasks requiring natural language generation. Developers looking for a Llama 2-based model with specific fine-tuning for generative tasks might find this model suitable.