RLHFlow/Llama3-SFT-v2.0-epoch2
RLHFlow/Llama3-SFT-v2.0-epoch2 is an 8 billion parameter instruction-tuned causal language model developed by RLHFlow, fine-tuned from Meta-Llama-3-8B. This model is the supervised fine-tuning (SFT) checkpoint used within the RLHFlow/Online-RLHF project, which focuses on advanced RLHF workflows. It was trained for two epochs on the RLHFlow/RLHFlow-SFT-Dataset-ver2, with a context length of 8192 tokens, making it suitable for research and applications in reinforcement learning from human feedback.
Loading preview...
Overview
RLHFlow/Llama3-SFT-v2.0-epoch2 is an 8 billion parameter supervised fine-tuning (SFT) checkpoint, derived from the meta-llama/Meta-Llama-3-8B base model. It was developed by RLHFlow as part of their Online-RLHF project, which explores advanced reinforcement learning from human feedback workflows. The model's training details, including a global batch size of 128 and a learning rate of 2e-5, are documented in the project's repository.
Key Capabilities
- Foundation for RLHF: Serves as the SFT base for subsequent online RLHF stages, as detailed in the associated paper.
- Instruction Following: Fine-tuned on the
RLHFlow/RLHFlow-SFT-Dataset-ver2for two epochs, enhancing its ability to follow instructions. - Context Handling: Supports a context length of 8192 tokens, allowing for processing longer inputs and generating more extensive responses.
Good For
- RLHF Research: Ideal for researchers and developers working on reinforcement learning from human feedback, particularly those interested in the RLHFlow framework.
- Instruction-tuned Applications: Suitable for general instruction-following tasks where an 8B parameter model with a substantial context window is beneficial.
- Further Fine-tuning: Can serve as a strong base for additional fine-tuning on specific downstream tasks or datasets.