princeton-nlp/Llama-3-Instruct-8B-KTO-v0.2
princeton-nlp/Llama-3-Instruct-8B-KTO-v0.2 is an 8 billion parameter instruction-tuned language model developed by princeton-nlp, based on the Llama 3 architecture. This model is specifically optimized using the KTO (Kahneman-Tversky Optimization) method, as detailed in the SimPO preprint, to enhance preference alignment without requiring a reference-free reward. It is designed for general instruction following tasks, leveraging its 8192 token context length for robust performance.
Loading preview...
Model Overview
princeton-nlp/Llama-3-Instruct-8B-KTO-v0.2 is an 8 billion parameter instruction-tuned model built upon the Llama 3 architecture. Developed by princeton-nlp, this model incorporates a novel optimization technique known as KTO (Kahneman-Tversky Optimization). KTO is a preference optimization method that aims to improve model alignment with human preferences, notably without the need for a reference-free reward mechanism, as described in the SimPO preprint.
Key Capabilities
- Instruction Following: Optimized for accurately understanding and executing a wide range of user instructions.
- Preference Alignment: Utilizes KTO for enhanced alignment with desired output characteristics and user preferences.
- Context Handling: Features an 8192 token context window, allowing for processing and generating longer sequences of text.
Good For
- Applications requiring robust instruction following.
- Research and development in preference optimization techniques, particularly KTO.
- Tasks benefiting from a model with improved alignment to human feedback without explicit reward modeling.