mlabonne/Gemmalpaca-2B
mlabonne/Gemmalpaca-2B is a 2.6 billion parameter language model based on the Gemma-2B architecture, fine-tuned on the Alpaca-GPT4 dataset. This model demonstrates improved performance over Google's gemma-2b-it on various benchmarks, including Nous' benchmark suite. With an 8192-token context length, it is optimized for instruction-following tasks and general conversational applications, particularly when used with the Alpaca chat template.
Loading preview...
Gemmalpaca-2B: An Enhanced Gemma-2B Variant
mlabonne/Gemmalpaca-2B is a 2.6 billion parameter model derived from the Gemma-2B base, specifically fine-tuned using the vicgalle/alpaca-gpt4 dataset. This supervised fine-tuning process has resulted in a model that outperforms Google's official gemma-2b-it (instruction-tuned) version on several key benchmarks.
Key Capabilities & Performance
- Superior Benchmark Scores: Achieves an average score of 38.39 on Nous' benchmark suite, surpassing
gemma-2b-it(36.1) andgemma-2b(34.26). Notable improvements are seen in AGIEval and GPT4All metrics. - Extended Context Window: Features an 8192-token context length, allowing for more extensive conversational turns and complex prompts.
- Instruction Following: Designed for instruction-following tasks, leveraging the Alpaca-GPT4 dataset for robust response generation.
Recommended Usage
- Alpaca Chat Template: It is recommended to use this model with the Alpaca chat template rather than the Gemma Instruct template for optimal performance.
- Stop Token: Ensure
</s>is added as a stop token for proper generation termination. - Quantized Versions: GGUF quantized models are available for efficient local deployment.
This model serves as a strong demonstration of effective fine-tuning on Gemma architecture, offering enhanced performance for general-purpose conversational AI in its size class.