jondurbin/bagel-34b-v0.4
jondurbin/bagel-34b-v0.4 is a 34 billion parameter language model fine-tuned from yi-34b-200k by jondurbin. This model is distinguished by its extensive and diverse SFT training data, encompassing reasoning, coding, multilingual reading comprehension, roleplay, function calling, and specialized prompt formats for tasks like context-obedient question answering and novel writing. It is designed for versatility across a wide range of instruction-following and creative generation tasks.
Loading preview...
jondurbin/bagel-34b-v0.4: A Versatile Instruction-Tuned Model
jondurbin/bagel-34b-v0.4 is a 34 billion parameter language model, fine-tuned from the yi-34b-200k base model. Developed by jondurbin, this model emphasizes broad instruction-following capabilities through a highly diverse Supervised Fine-Tuning (SFT) dataset. A key differentiator is its training across multiple prompt formats (Vicuna, Llama-2, Alpaca, ChatML) for each instruction, enhancing generalization.
Key Capabilities & Features
- Extensive Data Sources: Trained on a wide array of datasets covering reasoning (ai2_arc), coding (apps, python_alpaca, rosetta_code), multilingual reading (belebele), roleplay (bluemoon, limarp-augmented, cinematika), and specialized domains like biology, chemistry, math, and physics.
- Advanced Prompting Strategies: Supports unique prompt formats for:
- Context-Obedient Question Answering: Designed for RAG, it prioritizes provided context and minimizes hallucinations.
- Summarization: Optimized for generating concise summaries from input text.
- Function Calling: Integrates two distinct function-calling formats for tool use.
- Chain of Thought: Facilitates multi-step reasoning and solution ranking.
- Creative Writing: Includes specific formats for novel writing (chapter-by-chapter) and character card creation.
- Specialized Tasks: Boolean questions, SQL query generation, emotion detection (VAD scores), and multi-character chat direction.
- Decontamination: Employs cosine similarity decontamination against common benchmarks to ensure data integrity.
Ideal Use Cases
This model is well-suited for applications requiring a highly adaptable instruction-following LLM, particularly those involving complex reasoning, code generation, creative content generation, and structured data interaction (e.g., SQL, function calls). Its specialized prompt formats make it effective for RAG systems, summarization, and advanced conversational AI.