jondurbin/airoboros-13b-gpt4-1.3

TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kPublished:Jun 20, 2023License:cc-by-nc-4.0Architecture:Transformer Open Weights Cold

The jondurbin/airoboros-13b-gpt4-1.3 is a 13 billion parameter LLaMA-based model fine-tuned using QLoRA with completely synthetic training data generated by GPT-4. This iteration enhances coding instruction handling with 'PLAINFORMAT' versions and includes thousands of new Orca-style reasoning instructions. It is primarily intended for research use, focusing on improved reasoning and coding capabilities.

Loading preview...

Airoboros-13B-GPT4-1.3 Overview

This model is a QLoRA fine-tuned 13 billion parameter LLaMA model, developed by jondurbin, utilizing entirely synthetic training data generated by GPT-4. It represents an extension of previous Airoboros versions, with specific enhancements aimed at improving its utility.

Key Enhancements

  • Coding Instructions: All coding instructions now include an equivalent " PLAINFORMAT" version, designed to offer more structured output.
  • Reasoning Capabilities: Incorporates thousands of new Orca-style reasoning instructions, structured with reasoning presented before the final answer.
  • Diverse Content: Includes additional varied items, such as an initial attempt at multi-character interactions featuring asterisked actions and quoted speech.

Usage and Licensing

The model was fine-tuned using a modified version of QLoRA, which includes updates for compatibility with previous full fine-tune versions and uses a slightly modified Vicuna template. It is intended and licensed strictly for research use only under a custom license that incorporates the LLaMA research license and OpenAI's data usage clauses, prohibiting commercial application.

Finetunes
1 models