Model Overview

wandb/gemma-2b-zephyr-sft is a 2.5 billion parameter language model developed by wandb, building upon Google's Gemma 2B base model. It incorporates the Supervised Fine-Tuning (SFT) recipe from the Zephyr project, which involves training on a diverse mix of publicly available and synthetic datasets. This fine-tuning process aims to improve the model's ability to follow instructions and generate coherent, contextually relevant text.

Key Capabilities & Performance

This model is primarily designed for English language tasks. Its performance on the Open LLM Leaderboard indicates an average score of 47.18. Specific benchmark results include:

• AI2 Reasoning Challenge (25-Shot): 49.74
• HellaSwag (10-Shot): 72.38
• MMLU (5-Shot): 41.37
• TruthfulQA (0-shot): 34.42
• Winogrande (5-shot): 66.93
• GSM8k (5-shot): 18.27

Training Details

The model was trained using the alignment handbook recipe and logged to a Weights & Biases workspace. The training process was efficient, completing in approximately 2 hours on an 8xA100 80GB node provided by Lambda Labs.

Use Cases

Given its fine-tuned nature and performance metrics, this model is suitable for various general-purpose natural language processing tasks, particularly those requiring instruction following and conversational abilities. Its relatively small size (2.5B parameters) makes it a candidate for applications where computational resources are a consideration.