ZhiXi-13B-Diff-FP16: A Knowledge-Enhanced LLaMA Model
ZhiXi-13B-Diff-FP16 is a 13 billion parameter language model developed by ZJUNLP, built upon the LLaMA architecture. This model focuses on addressing challenges in knowledge acquisition and comprehension within large language models, such as knowledge updating difficulties and potential discrepancies.
Key Capabilities
- Enhanced Chinese Understanding: Achieved through full-scale pre-training on extensive Chinese corpora (Baidu Baike, Wudao, Chinese Wikipedia), while retaining original English and code capacities.
- Information Extraction (IE): Optimized for NER, RE, and IE tasks using a KG2Instructions approach and fine-tuning with a 1.4 million Chinese instruction dataset.
- Instruction Following: Demonstrates strong instruction-following abilities in both Chinese and English, including translation, coding, and general knowledge queries.
- Bilingual Support: Capable of handling tasks and generating text in both Chinese and English.
What Makes It Different
Unlike many LLMs, ZhiXi-13B-Diff-FP16 specifically integrates robust Chinese understanding and knowledge extraction capabilities into the LLaMA framework through a unique pre-training and instruction-tuning process. It aims to provide more accurate and comprehensive extraction results compared to other large models, as highlighted in its information extraction cases.
Good For
- Applications requiring strong Chinese language processing and knowledge extraction.
- Tasks involving Named Entity Recognition (NER), Relation Extraction (RE), and Information Extraction (IE).
- Developers seeking a LLaMA-based model with enhanced bilingual instruction-following and coding abilities.