jondurbin/airoboros-7b-gpt4-1.1
The jondurbin/airoboros-7b-gpt4-1.1 is a 7 billion parameter Llama-based language model fine-tuned by jondurbin, featuring a 4096-token context length. It is specifically optimized for context-obedient question answering, coding, and general reasoning tasks, utilizing a completely synthetic training dataset generated by GPT-4. This model excels at adhering to provided context and reducing hallucinations, making it suitable for applications requiring precise information extraction.
Loading preview...
Overview
jondurbin/airoboros-7b-gpt4-1.1 is a 7 billion parameter Llama model, fine-tuned using a unique, entirely synthetic dataset generated by GPT-4. This model is an update to the original airoboros-7b-gpt4, incorporating approximately 1,000 additional coding instructions and improvements to context handling. It is designed to be highly context-obedient, meaning it prioritizes information from provided context over its pre-trained knowledge, significantly reducing hallucinations.
Key Capabilities
- Context-Obedient Question Answering: Trained to strictly adhere to provided context, using a specific
BEGININPUT/BEGINCONTEXT/BEGININSTRUCTIONformat to ensure accurate, source-limited responses. - Coding: Enhanced with additional coding instructions, demonstrating proficiency in generating code for various requirements, including complex server implementations.
- Reasoning & Trivia: Shows capabilities in mathematical reasoning (though noted as still improving), trivia, and multiple-choice questions.
- Creative Writing: Can generate diverse text, including creative formats like a pirate-themed resignation letter.
Good For
- Applications requiring precise information extraction from provided documents or data.
- Developing code generation tools or assistants.
- Tasks where reducing model hallucination is critical by enforcing context adherence.
- Research into synthetic data training methods and their impact on model behavior.