Zenith-7B Model Overview

Zenith-7B is a 7 billion parameter instruction-tuned language model developed by Xenon1, built upon the Mistral-7B-v0.1 architecture. This model distinguishes itself through its fine-tuning process, which utilized the Ultrafeedback dataset and incorporated techniques described in the "Self-Rewarding Language Models" paper. This approach aims to enhance the model's ability to understand and respond to instructions effectively.

Key Capabilities

• Instruction Following: Optimized for accurately interpreting and executing user instructions.
• Conversational AI: Designed to engage in multi-turn conversations, maintaining context and coherence.
• Mistral-7B Architecture: Inherits efficient architectural features from Mistral-7B-v0.1, including Grouped-Query Attention, Sliding-Window Attention, and a Byte-fallback BPE tokenizer.

Instruction Format

Zenith-7B requires a specific instruction format using [INST] and [/INST] tokens for optimal performance. The first instruction in a conversation should begin with a begin-of-sentence ID, while subsequent instructions do not. This format is supported via Hugging Face's apply_chat_template() method, simplifying integration for developers.

Good For

• Applications requiring robust instruction following.
• Building chatbots and conversational agents.
• Tasks benefiting from a model fine-tuned with self-rewarding techniques for improved response quality.