Overview

Fizzarolli/sappha-2b-v3 is an instruction-tuned model based on the Gemma-2B architecture, fine-tuned using QLoRA with Unsloth. This iteration, following a private v2 failure, aims to provide a more stable and capable small language model.

Key Capabilities & Performance

This model shows competitive performance against its base model, gemma-2b-it, and dolphin-2.8-gemma-2b across several benchmarks:

• MMLU (five-shot): Achieves 38.02, outperforming both comparison models.
• HellaSwag (zero-shot): Scores 51.70, also surpassing its counterparts.
• PIQA (one-shot): Reaches 75.46, indicating strong common sense reasoning.
• TruthfulQA (zero-shot): While slightly lower than gemma-2b-it, it remains competitive.

Prompt Format

The model utilizes a basic ChatML format for interactions, supporting clear system, user, and assistant turns.

Good For

• General instruction-following tasks where a compact model size (2.5B parameters) is beneficial.
• Applications requiring a model with an 8192-token context length.
• Scenarios where improved benchmark performance over the base Gemma-2B model is desired.