jondurbin/airoboros-13b-gpt4-1.2
jondurbin/airoboros-13b-gpt4-1.2 is a 13 billion parameter LLaMa model fine-tuned by jondurbin using QLoRA. It is trained on completely synthetic data generated by GPT-4, focusing on coding, math/reasoning, trivia, and role-playing. This model excels at generating context-obedient responses and includes specific updates for plain-text code output.
Loading preview...
Model Overview
jondurbin/airoboros-13b-gpt4-1.2 is a 13 billion parameter LLaMa model, fine-tuned using QLoRA with a dataset entirely generated by GPT-4. This version extends the previous 1.1 release with thousands of new training data points and introduces a "PLAINFORMAT" option for coding prompts to output code without markdown or explanations.
Key Capabilities
- Synthetic Data Training: Fine-tuned on a diverse dataset created by GPT-4, emphasizing quality and breadth.
- Enhanced Coding: Includes numerous coding examples across various languages and libraries, with a specific feature for plain-text code output.
- Reasoning and Math: Incorporates ORCA-style ELI5 instruction/response pairs for improved mathematical and reasoning abilities.
- Diverse Applications: Covers trivia, role-playing, multiple-choice, fill-in-the-blank, and context-obedient question answering.
- Vicuna Template: Utilizes a modified Vicuna template for consistent chat interactions.
Usage and Licensing
This model is intended for research use only due to its reliance on the LLaMa base model license and OpenAI's terms for data generation. Commercial use is prohibited. Quantized versions are available from TheBloke for easier deployment.