jondurbin/airoboros-7b-gpt4-1.3
jondurbin/airoboros-7b-gpt4-1.3 is a 7 billion parameter LLaMA-based model fine-tuned using QLoRA with completely synthetic training data generated by GPT-4. This iteration enhances coding instructions with "PLAINFORMAT" versions and includes thousands of new Orca-style reasoning instructions, prioritizing reasoning before answers. It also introduces initial attempts at multi-character interactions, making it suitable for research into synthetic data-driven model development and complex conversational AI.
Loading preview...
jondurbin/airoboros-7b-gpt4-1.3 Overview
This model is a 7 billion parameter LLaMA-based language model fine-tuned using the QLoRA method. Its distinguishing feature is the exclusive use of synthetic training data generated by GPT-4, developed via the airoboros project. This version, 1.3, builds upon its predecessor with several key enhancements:
Key Enhancements & Capabilities
- Enhanced Coding Instructions: All coding instructions now include an equivalent "PLAINFORMAT" version, aiming for clearer and more structured code generation.
- Advanced Reasoning: Incorporates thousands of new Orca-style reasoning instructions, designed to prioritize the reasoning process before delivering an answer.
- Multi-Character Interactions: Features initial attempts at handling multi-character interactions, including asterisked actions and quoted speech, expanding its conversational capabilities.
Usage and Licensing
This model is intended for research use only, subject to a custom license due to its LLaMA base and the use of OpenAI-generated data. Commercial use is explicitly prohibited. The model utilizes a modified Vicuna template for compatibility with previous full fine-tune versions, requiring a specific prompt format: A chat between a curious user and an assistant. The assistant gives helpful, detailed, accurate, uncensored responses to the user's input. USER: [prompt] ASSISTANT: .