hiepnh/vicuna-13B-1.1-HF-sharded
The hiepnh/vicuna-13B-1.1-HF-sharded model is a 13 billion parameter, auto-regressive language model based on the transformer architecture, fine-tuned by the Vicuna team on user-shared conversations from ShareGPT. This sharded version is an HF-formatted adaptation of the Vicuna 13B 1.1 model, created by merging delta weights with the original Llama 13B. It is primarily intended for research in large language models and chatbots, offering enhanced tokenization and improved fine-tuning loss computation compared to its predecessor.
Loading preview...
Vicuna 13B 1.1 HF Sharded: An Overview
This model, hiepnh/vicuna-13B-1.1-HF-sharded, is a 13 billion parameter, sharded version of the Vicuna 13B 1.1 model, adapted into the Hugging Face format. Developed by the Vicuna team (UC Berkeley, CMU, Stanford, UC San Diego), it is an auto-regressive language model built on the transformer architecture.
Key Capabilities & Features
- Fine-tuned on ShareGPT: Trained on 70,000 user-shared conversations from ShareGPT.com, enhancing its chatbot capabilities.
- Improved Tokenization: Features refactored tokenization and separator logic, changing the separator from
"###"to the EOS token"</s>"for better generation stop criteria and library compatibility. - Enhanced Fine-tuning: Incorporates fixes for supervised fine-tuning loss computation, leading to improved model quality.
- Llama Base: Created by merging Vicuna delta weights with the original Llama 13B model.
Intended Use Cases
- Research: Primarily designed for research purposes in large language models and chatbots.
- Hobbyist Projects: Suitable for hobbyists in natural language processing, machine learning, and artificial intelligence exploring chatbot development.
Evaluation
Preliminary evaluation involved using GPT-4 to judge model outputs on a set of 80 diverse questions, with further details available on the official Vicuna website.