Model Overview
jondurbin/bagel-7b-v0.4 is a 7 billion parameter model built upon the Mistral-7b architecture, developed by jondurbin. This version is the pre-DPO (Direct Preference Optimization) iteration, specifically noted for its potential suitability in roleplay scenarios.
Key Capabilities
This model is distinguished by its extensive fine-tuning across a broad spectrum of tasks, utilizing numerous SFT (Supervised Fine-Tuning) data sources. Notable capabilities include:
- Advanced Reasoning: Trained on datasets like ai2_arc, boolq, and mathinstruct for improved logical and mathematical problem-solving.
- Code Generation: Incorporates datasets such as codeparrot/apps and rosetta_code for Python and multi-language code tasks.
- Function Calling: Supports two primary formats for function calling, including GlaiveAI's method, enabling integration with external tools.
- Context-Obedient Question Answering: Designed to answer questions strictly from provided context, reducing hallucinations in RAG applications.
- Summarization: Optimized for various summarization tasks, including conversational memory creation.
- Creative Writing & Roleplay: Enhanced with datasets like cinematika and bluemoon for generating engaging narratives and character interactions.
- Multi-format Prompting: Uniquely trained with four prompt formats (Vicuna, Llama-2, Alpaca, ChatML) for each instruction, improving generalization and adaptability.
Good For
- Roleplay and Creative Writing: Excels in generating dynamic and nuanced character interactions and novel-style content.
- Complex Instruction Following: Capable of handling intricate instructions, including chain-of-thought reasoning and reWOO-style function planning.
- Structured Output Generation: Proficient in producing JSON for emotion detection, character cards, and SQL queries.
- RAG Applications: Its context-obedient QA and summarization features make it suitable for retrieval-augmented generation systems.