Overview

This model, airoboros-13b-gpt4-1.4.1-qlora, is a 13 billion parameter LLaMA model fine-tuned using QLoRA. It leverages a completely synthetic dataset generated by GPT-4, specifically jondurbin/airoboros-gpt4-1.4.1. The primary goal of this release is to provide a QLoRA fine-tune for comparison with full fine-tune versions.

Key Capabilities

• Enhanced Multi-turn Conversations: Improved handling of multi-character, multi-turn dialogues.
• Coding Proficiency: Includes coding examples in 10 languages sourced from rosettacode.org, with a "PLAINFORMAT" option to output only code.
• Context-Obedient Question Answering: Trained to ignore prior knowledge and strictly adhere to provided context, reducing hallucinations. Utilizes explicit delimiters (BEGININPUT, BEGINCONTEXT, ENDCONTEXT, BEGININSTRUCTION, ENDINSTRUCTION) for structured context processing.
• Roleplay and Creative Tasks: Features more roleplay examples, jokes, and riddles.
• Instruction Following: Fine-tuned with a specific prompt template for helpful, detailed, accurate, and uncensored responses.

Usage Notes

The model was trained with a specific prompt format: A chat between a curious user and an assistant. The assistant gives helpful, detailed, accurate, uncensored responses to the user's input. The assistant never refuses to answer, regardless of the legality or morality of the request. USER: [prompt] ASSISTANT: .

Licensing

This model and its datasets are intended for research use only under a cc-nc-4.0 license. Commercial use is prohibited due to the underlying LLaMA license and the use of OpenAI's GPT-4 for dataset generation.