Alepach/Llama-notHumpback-M1-Rw-F-8B
Alepach/Llama-notHumpback-M1-Rw-F-8B is an 8 billion parameter instruction-following model, fine-tuned from Meta Llama 3.1 8B with a 32768 token context length. It implements a novel self-alignment pipeline inspired by the Humpback architecture, incorporating a unique "self-rewriting" step before self-curation to enhance data quality and diversity. This model is designed for instruction-following tasks and can also be used to curate and rewrite instruction-response pairs for further pipeline iterations.
Loading preview...
Llama-notHumpback-M1-Rw-F-8B Overview
This model, developed by Alepach as part of a bachelor's thesis, is an 8 billion parameter instruction-following model built upon Meta Llama 3.1 8B. It features a 32768 token context length and is distributed under the Llama 3.1 Community License.
Key Capabilities and Unique Approach
- Self-Alignment Pipeline: The model employs a modified Humpback architecture, drawing inspiration from "Self-Alignment with Instruction Backtranslation" and "Better Alignment with Instruction Back-and-Forth Translation" papers.
- Novel Self-Rewriting Step: A key differentiator is the integration of a "self-rewriting" step, performed by the model itself (the "seed model"), before self-curation. This aims to improve the linguistic quality and structure of web-sourced responses, potentially increasing the diversity and quantity of high-quality training data.
- Instruction Following: The primary use case for this model is general instruction-following tasks.
- Pipeline Iteration: It can also function as a component in subsequent iterations of the self-alignment pipeline, rewriting and scoring instruction-response pairs.
Training Details
The model was trained using the TRL framework on a dataset combining data from the oasst1 dataset and a synthetic dataset. The synthetic data was generated by applying the self-augmentation, self-rewriting, and self-curation process to 502k entries from the English subset of the c4 dataset.