project-baize/baize-v2-13b
Baize-v2-13b is a 13 billion parameter open-source chat model developed by UCSD and Sun Yat-Sen University, fine-tuned with LoRA using supervised fine-tuning (SFT) and self-distillation with feedback (SDF). This model is designed for conversational AI, providing detailed responses in a specific chat format. It is merged with LLaMA, making it ready for direct use in chat applications.
Loading preview...
Baize-v2-13b: An Open-Source Chat Model
Baize-v2-13b is a 13 billion parameter open-source chat model developed by UCSD and Sun Yat-Sen University. It is fine-tuned using LoRA with a combination of supervised fine-tuning (SFT) and self-distillation with feedback (SDF) techniques. The model is named after Baize, a mythical Chinese creature known for its extensive knowledge and ability to speak human languages, reflecting the developers' aspirations for its conversational capabilities.
Key Capabilities & Features
- Conversational AI: Specifically designed and fine-tuned for chat-based interactions.
- Detailed Responses: Engineered to provide comprehensive and detailed answers.
- Specific Chat Format: Requires a precise input format (
[|Human|]and[|AI|]) for optimal performance, ensuring structured dialogue. - Ethical Guidelines: Programmed to decline engagement with unethical, controversial, or sensitive topics.
- LLaMA Integration: The checkpoint is merged with LLaMA, making it readily usable.
Good For
- Developers building chat applications requiring a structured conversational model.
- Research into self-distillation and LoRA fine-tuning methods for large language models.
- Creating AI assistants that provide detailed, markdown-formatted responses while adhering to ethical content guidelines. More details and usage instructions can be found on the Project Baize GitHub and in their research paper.