Spico/Humback-M0
TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kLicense:apache-2.0Architecture:Transformer0.0K Open Weights Cold
Spico/Humback-M0 is a 7 billion parameter supervised fine-tuning (SFT) model developed by Spico. This model is part of the Humback framework, designed to augment instruction data with high quality for fine-tuning. Trained on a sampled dataset from oasst1, Humback-M0 serves as an initial SFT model for reproduction of the Humback methodology. It is primarily intended for research and development in instruction data augmentation techniques.
Loading preview...
Humback-M0 Overview
Spico/Humback-M0 is a 7 billion parameter supervised fine-tuning (SFT) model, representing the initial $M_0$ model within the novel Humback framework. The Humback framework focuses on augmenting instruction data to achieve high quality for subsequent fine-tuning processes.
Key Capabilities & Characteristics
- Instruction Data Augmentation: Humback-M0 is a foundational component for the Humback framework, which aims to improve the quality of instruction data used in SFT.
- Seed Data Training: This model was trained using a seed dataset, specifically a sampled subset derived from the oasst1 dataset.
- Research Reproduction: It serves as a reproduction model for the methodology detailed in the Humback research paper, focusing on self-alignment with instruction backtranslation.
Good For
- Research into Instruction Tuning: Ideal for researchers exploring methods for enhancing instruction data quality and self-alignment techniques.
- Reproducing Humback Framework: Useful for developers and researchers looking to understand and replicate the Humback framework's initial SFT stage.
- Experimental SFT: Provides a base model for experimenting with instruction data augmentation strategies.