jondurbin/airoboros-13b-gpt4-1.1
jondurbin/airoboros-13b-gpt4-1.1 is a 13 billion parameter Llama-based language model fine-tuned by jondurbin. It utilizes completely synthetic training data generated by GPT-4, focusing on diverse tasks including trivia, math, coding, and context-obedient question answering. This model excels at adhering to provided context for question answering and demonstrates improved coding capabilities.
Loading preview...
Overview
jondurbin/airoboros-13b-gpt4-1.1 is a 13 billion parameter Llama model, an update to the original airoboros-13b-gpt4, incorporating approximately 1,000 additional coding instructions and refined context instructions. This model was fine-tuned using entirely synthetic data generated by GPT-4, with a specific emphasis on improving its ability to follow context and reduce hallucinations.
Key Capabilities
- Context-Obedient Question Answering: Designed to prioritize provided context over its internal knowledge, significantly reducing hallucinations. It uses a specific delimited format (
BEGININPUT,BEGINCONTEXT,BEGININSTRUCTION) for closed-context queries. - Improved Coding: Enhanced with additional coding instructions, demonstrating proficiency in generating code for various requirements, such as FastAPI web servers and multi-threaded TCP servers in C.
- Diverse Task Handling: Trained on a broad dataset covering trivia, math/reasoning, multiple-choice questions, and general knowledge.
- Theory of Mind: Includes training for tasks related to understanding and responding to mental states.
Usage and Licensing
This model is intended for research use only due to its Llama base and the use of OpenAI-generated data. Commercial use is explicitly prohibited by its 'cc-nc-4.0' license and underlying terms.