jondurbin/airoboros-l2-70b-2.2.1
The jondurbin/airoboros-l2-70b-2.2.1 is a 69 billion parameter experimental language model, fine-tuned by jondurbin primarily using synthetic data generated by the Airoboros project. This iteration focuses heavily on instruction following, rather than casual chat or roleplay, and includes updates such as re-generated writing responses, longer contextual blocks, and de-censoring. It excels in structured tasks like context-obedient question answering, summarization, code generation, agent/function calling, and chain-of-thought reasoning.
Loading preview...
Overview
jondurbin/airoboros-l2-70b-2.2.1 is a 69 billion parameter experimental model, building upon airoboros-l2-70b-2.2. It is primarily fine-tuned with synthetic data generated by the airoboros project, with a strong emphasis on instruction following over casual chat or roleplay.
Key Updates & Features
This version incorporates several key changes:
- Improved Writing Responses: Re-generated writing responses for enhanced quality.
- Extended Context: Utilizes longer contextual blocks during training.
- Data Refinements: Removal of "rp" (roleplay) data and less aggressive de-censoring.
- Training Epochs: Trained for 4 epochs, an increase from the previous 3.
- Instruction Following: Designed to excel at understanding and executing complex instructions.
Capabilities
The model supports a variety of advanced use cases:
- Context-Obedient QA: Trained to answer questions strictly based on provided context, minimizing hallucinations, using a specific
BEGININPUT/BEGINCONTEXT/BEGININSTRUCTIONformat. - Summarization: Capable of summarizing text, with specific formatting for input.
- Code Generation: Generates code for complex requirements, supporting inline criteria and a
PLAINFORMAToption for raw code output. - Agent/Function Calling: Generates JSON or YAML for function calls based on natural language input, similar to OpenAI's function calling.
- Chain-of-Thought Reasoning: Can provide multiple potential solutions, rank them, and select the most feasible answer for complex problems.
- reWOO-style Execution Planning: Supports generating systematic plans for multi-tool execution, outputting a sequence of function calls and evidence references.
Important Notes
- Quantization Warning: Q4_0 quantization is explicitly stated to produce garbage results with this version.
- Prompt Format: Uses a
USER: {prompt}\nASSISTANT:format, with suggestions for adding stopping criteria. - Licensing: Built on Llama-2/CodeLlama with Meta's custom license. The fine-tuning data, generated via OpenAI API, introduces ambiguity regarding commercial use due to OpenAI's ToS. Users are advised to exercise caution for commercial applications.