Xenon1/Zenith-7B
Zenith-7B is a 7 billion parameter causal language model developed by Xenon1, fine-tuned from Mistral-7B-v0.1. This model leverages techniques from the "Self-Rewarding Language Models" paper, specifically using the Ultrafeedback dataset for instruction tuning. It is optimized for following instructions and engaging in conversational tasks, making it suitable for various interactive AI applications.
Loading preview...
Zenith-7B Model Overview
Zenith-7B is a 7 billion parameter instruction-tuned language model developed by Xenon1, built upon the Mistral-7B-v0.1 architecture. This model distinguishes itself through its fine-tuning process, which utilized the Ultrafeedback dataset and incorporated techniques described in the "Self-Rewarding Language Models" paper. This approach aims to enhance the model's ability to understand and respond to instructions effectively.
Key Capabilities
- Instruction Following: Optimized for accurately interpreting and executing user instructions.
- Conversational AI: Designed to engage in multi-turn conversations, maintaining context and coherence.
- Mistral-7B Architecture: Inherits efficient architectural features from Mistral-7B-v0.1, including Grouped-Query Attention, Sliding-Window Attention, and a Byte-fallback BPE tokenizer.
Instruction Format
Zenith-7B requires a specific instruction format using [INST] and [/INST] tokens for optimal performance. The first instruction in a conversation should begin with a begin-of-sentence ID, while subsequent instructions do not. This format is supported via Hugging Face's apply_chat_template() method, simplifying integration for developers.
Good For
- Applications requiring robust instruction following.
- Building chatbots and conversational agents.
- Tasks benefiting from a model fine-tuned with self-rewarding techniques for improved response quality.