Humback-Myx: A Backward Model for Instruction Data Augmentation

Spico/Humback-Myx is a 7 billion parameter model specifically engineered as a "backward model" ($M_{yx}$) within the Humback framework. Its core purpose is to augment instruction data for supervised fine-tuning by generating instructions from given responses.

Key Capabilities

• Instruction Backtranslation: Trained to reverse the typical instruction-response generation, producing instructions from provided answers.
• Data Augmentation: Designed to enhance the quality and quantity of instruction data for training other language models.
• OASST1 Seed Data: Utilizes a sampled dataset from oasst1 for its reversed-order training.

What Makes This Different?

Unlike traditional instruction-tuned models that generate responses from instructions, Humback-Myx operates in reverse. This unique approach allows for the creation of high-quality synthetic instruction-response pairs, which can then be used to improve the performance of forward-facing instruction-following models. It is a specialized tool for researchers and developers focused on advanced data augmentation techniques for LLM training.

Good For

• Generating synthetic instruction data for supervised fine-tuning.
• Research into self-alignment and instruction backtranslation methods.
• Improving the robustness and diversity of training datasets for instruction-following models.