Xinging/llama2-7b_sft_0.3_ratio_alpaca_gpt4_proj_by_tydiqa_ntrain_49400_default
The Xinging/llama2-7b_sft_0.3_ratio_alpaca_gpt4_proj_by_tydiqa_ntrain_49400_default is a 7 billion parameter language model fine-tuned from Meta's Llama-2-7b-hf. This model was specifically fine-tuned on a dataset derived from Alpaca, GPT-4 projections, and TyDiQA, suggesting an optimization for instruction-following and question-answering tasks. It is designed for general language generation and understanding, leveraging its Llama 2 base with specialized instruction tuning.
Loading preview...
Overview
This model, llama2-7b_sft_0.3_ratio_alpaca_gpt4_proj_by_tydiqa_ntrain_49400_default, is a 7 billion parameter language model built upon the foundational Meta Llama-2-7b-hf architecture. It has undergone supervised fine-tuning (SFT) using a unique dataset that combines elements from Alpaca, GPT-4 projections, and the TyDiQA dataset. This specific training regimen aims to enhance its capabilities in instruction-following and diverse question-answering scenarios.
Key Training Details
- Base Model: meta-llama/Llama-2-7b-hf
- Fine-tuning Dataset: A composite dataset referred to as
0.3_ratio_alpaca_gpt4_proj_by_tydiqa_ntrain_49400. - Hyperparameters:
- Learning Rate: 2e-05
- Batch Size: 32 (train), 8 (eval)
- Optimizer: AdamW with specific betas and epsilon
- LR Scheduler: Cosine with 0.03 warmup ratio
- Epochs: 3.0
Potential Use Cases
Given its fine-tuning on instruction-following and question-answering data, this model is likely suitable for:
- Instruction-following tasks: Generating responses based on explicit instructions.
- General question answering: Providing informative answers to a wide range of queries.
- Text generation: Creating coherent and contextually relevant text.