UCLA-AGI/zephyr-7b-sft-full-SPIN-iter1
UCLA-AGI/zephyr-7b-sft-full-SPIN-iter1 is a 7 billion parameter GPT-like language model developed by UCLA-AGI, fine-tuned using a self-play approach. Based on Mistral-7B-v0.1, this model leverages synthetic data from Ultrachat_200k to enhance its capabilities. It is specifically designed to improve performance through iterative self-play fine-tuning, aiming to convert weaker language models into stronger ones.
Loading preview...
Model Overview
UCLA-AGI/zephyr-7b-sft-full-SPIN-iter1 is a 7 billion parameter GPT-like model developed by UCLA-AGI. It is the first iteration of a self-play fine-tuned model, building upon the alignment-handbook/zephyr-7b-sft-full base, which itself is derived from mistralai/Mistral-7B-v0.1. The model's unique training methodology involves self-play fine-tuning using synthetic data generated from the HuggingFaceH4/ultrachat_200k dataset, as detailed in the paper "Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models" (arXiv:2401.01335).
Key Capabilities & Performance
This model is primarily focused on general language understanding and generation in English. Its performance on the Open LLM Leaderboard indicates a balanced capability across various benchmarks:
- Average Score: 62.86
- ARC (25-shot): 65.87
- HellaSwag (10-shot): 85.44
- MMLU (5-shot): 60.95
- TruthfulQA (0-shot): 57.39
- Winogrande (5-shot): 76.64
- GSM8K (5-shot): 30.86
Training Methodology
The model was trained with a learning rate of 5e-07, a batch size of 8 across 8 GPUs (total batch size 64), using the RMSProp optimizer over 2 epochs. This self-play fine-tuning approach aims to iteratively refine the model's responses by generating and evaluating its own outputs, a method designed to enhance model strength from a weaker base.