KBlueLeaf/guanaco-7B-leh
KBlueLeaf/guanaco-7B-leh is a 7 billion parameter multilingual instruction-following language model based on the LLaMA architecture. It was fine-tuned using a modified Alpaca-LoRA method, specifically training the LoRA, embed_tokens, and lm_head components. This approach enhances its performance in Chinese and Japanese, making it more effective for instruction-based prompts and chatbot applications in these languages compared to the original LLaMA model. Its primary use case is as a multilingual chatbot, particularly strong in East Asian languages.
Loading preview...
Guanaco: Multilingual Instruction-Following LLaMA 7B
KBlueLeaf/guanaco-7B-leh is a 7 billion parameter language model built upon the LLaMA architecture, specifically fine-tuned for enhanced multilingual instruction-following capabilities. It leverages a modified alpaca-lora training approach, focusing on training the LoRA, embed_tokens, and lm_head components.
Key Capabilities
- Multilingual Performance: Significantly improved performance in Chinese and Japanese compared to the base LLaMA model, making it suitable for East Asian language applications.
- Instruction Following: Designed to follow instructions effectively, enabling more natural and useful interactions.
- Chatbot Functionality: Can be readily used as a chatbot, supporting conversational use cases with a specific prompt format.
- Reduced Memorization: The LoRA-based fine-tuning for attention parts (ignoring MLP) helps mitigate overfitting and memorization issues often seen in full fine-tuning of large LLaMA models.
Good for
- Multilingual Chatbots: Ideal for building chatbots that need to interact in Chinese, Japanese, and English.
- Instruction-Based Tasks: Performing tasks that require understanding and following explicit instructions.
- Experimentation: Developers looking to experiment with LoRA-based fine-tuning on LLaMA for multilingual enhancements.
This model utilizes datasets from alpaca-lora (cleaned Alpaca) and guanaco. Users can try the model via a provided Colab notebook or integrate it into Gradio web UIs using the associated GitHub repository.