RLHFlow/Llama3-SFT-v2.0-epoch3
RLHFlow/Llama3-SFT-v2.0-epoch3 is an 8 billion parameter instruction-tuned causal language model, fine-tuned from Meta-Llama-3-8B. Developed by RLHFlow, this model is the supervised fine-tuning checkpoint used within the RLHFlow/Online-RLHF project, which focuses on advanced RLHF workflows. It was trained for 3 epochs on the RLHFlow/RLHFlow-SFT-Dataset-ver2, utilizing a global batch size of 128 and packing samples into 8192-token chunks. This model is primarily intended for research and development in reinforcement learning from human feedback (RLHF) systems.
Loading preview...
Overview
RLHFlow/Llama3-SFT-v2.0-epoch3 is an 8 billion parameter supervised fine-tuning (SFT) checkpoint derived from the Meta-Llama-3-8B model. It serves as a foundational component within the RLHFlow/Online-RLHF project, which explores advanced reinforcement learning from human feedback (RLHF) workflows.
Key Training Details
- Base Model: Fine-tuned from meta-llama/Meta-Llama-3-8B.
- Dataset: Trained on the RLHFlow/RLHFlow-SFT-Dataset-ver2.
- Epochs: Underwent 3 training epochs.
- Parameters: Utilized a global batch size of 128 and a learning rate of 2e-5.
- Context Handling: Samples were packed and split into chunks of 8192 tokens, aligning with the model's context length.
Research Context
This model is directly associated with the research paper "RLHF Workflow: From Reward Modeling to Online RLHF" published in TMLR, 2024. It represents a specific stage in the development of more sophisticated RLHF methodologies.
Intended Use
This model is primarily designed for researchers and developers working on:
- Exploring and replicating the RLHF workflows detailed in the associated paper.
- Further fine-tuning or experimentation within the RLHF domain.
- Understanding the impact of specific SFT stages on overall RLHF performance.