BELLE-2/BELLE-Llama2-13B-chat-0.4M Overview
This model is a 13 billion parameter large language model, developed by BELLEGroup, that has been fine-tuned from the original Llama2-13B-chat. The fine-tuning process involved using 0.4 million Chinese instruction data, with the primary goal of enhancing the model's ability to produce stable and high-quality Chinese language outputs. The developers note that the base Llama2-chat model already performs commendably after Supervised Fine-Tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF), and this iteration aims to further specialize it for Chinese language tasks.
Key Capabilities
- Enhanced Chinese Language Generation: Optimized for generating high-quality and stable responses in Chinese.
- Instruction Following: Fine-tuned with Chinese instruction data to better adhere to given prompts.
- Llama2-chat Foundation: Benefits from the strong base performance of the Llama2-chat architecture.
Limitations and Considerations
- Factual Accuracy: May generate factual errors when responding to fact-related instructions.
- Harmful Content: Occasionally struggles to identify and avoid generating harmful responses.
- Reasoning and Coding: Requires further improvements in reasoning abilities and code generation.
This model is intended for research purposes only, and commercial use or other potentially harmful applications are not permitted due to its existing limitations.