Xenon1/Xenon-3
Xenon1/Xenon-3 is a 7 billion parameter instruction-tuned language model based on the Mistral-7B-v0.1 architecture, fine-tuned on the Ultrafeedback dataset. It leverages techniques from the "Self-Rewarding Language Models" paper to enhance its conversational abilities. This model is optimized for generating coherent and contextually relevant responses in instruction-following scenarios, supporting a context length of 8192 tokens.
Loading preview...
Xenon-3: Instruction-Tuned Mistral-7B-v0.1
Xenon-3 is a 7 billion parameter language model developed by Xenon1, built upon the robust Mistral-7B-v0.1 architecture. This model has undergone instruction fine-tuning using the Ultrafeedback dataset, incorporating methodologies outlined in the Self-Rewarding Language Models paper.
Key Capabilities & Features
- Instruction Following: Designed to accurately interpret and respond to user instructions, making it suitable for conversational AI and task-oriented dialogues.
- Mistral-7B-v0.1 Base: Inherits architectural strengths like Grouped-Query Attention, Sliding-Window Attention, and a Byte-fallback BPE tokenizer, contributing to efficient processing and performance.
- Context Length: Supports an 8192-token context window, allowing for more extended and complex interactions.
- Chat Template Support: Utilizes a specific instruction format (
[INST]...[/INST]) that is compatible with Hugging Face'sapply_chat_template()for seamless integration into applications.
When to Use Xenon-3
- Conversational Agents: Ideal for chatbots and virtual assistants requiring nuanced instruction adherence.
- Content Generation: Can be used for generating text based on specific prompts or scenarios.
- Research & Experimentation: A solid base for further fine-tuning or exploring self-rewarding learning techniques.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.