Airoboros-L2-7B-2.1: Instruction-Tuned Llama-2 Model
This model is an instruction fine-tuned Llama-2 variant, developed by jondurbin, leveraging synthetic data generated by the Airoboros framework. It introduces several experimental features aimed at enhancing conversational and creative generation capabilities.
Key Capabilities & Features
- Experimental Instruction Sets: Includes unique 'RP' (role-play) for multi-round chats with character cards and 'GTKM' for character-driven Q&A, designed to test alternatives to ghost attention.
- Enhanced Writing: Supports longer, more detailed writing prompts and next-chapter generation, with training data including "stylized_response" for better adherence to system card styles.
- Context-Obedient QA: Tuned to ignore prior knowledge and strictly use provided context for question answering, reducing hallucinations with a specific
BEGININPUT/BEGINCONTEXT formatting. - Coding & Function Calling: Capable of generating complex code based on requirements and supports agent/function calling with JSON or YAML output, similar to OpenAI's function calling.
- Chain-of-Thought & Execution Planning: Can generate multiple potential solutions with ranking for complex problems and supports reWOO-style execution planning for multi-tool tasks.
- "De-alignment": Includes a small dataset to reduce censorship from the base models, aiming for broader applicability.
Important Note
This version has a known prompt formatting bug in the training code, which will be addressed in version 2.2. Due to its reliance on OpenAI API-generated data, commercial use may be restricted by OpenAI's Terms of Service.