euclaise/ReMask-3B
euclaise/ReMask-3B is a 2.2 billion parameter language model developed by euclaise, fine-tuned using the novel ReMask regularization method. This method addresses exposure bias by penalizing deviations between masked and full sequence predictions, enhancing generative performance. ReMask-3B demonstrates improved capabilities on generative benchmarks like GSM8K, making it suitable for tasks requiring robust sequence generation.
Loading preview...
ReMask-3B: Enhancing Generative Performance through Regularized Masking
ReMask-3B is a 2.2 billion parameter model developed by euclaise, utilizing a novel finetuning approach called ReMask. This method aims to mitigate "exposure bias" commonly found in autoregressive models, where models struggle to maintain coherence over long generations due to predicting on their own outputs rather than ground truth.
Key Innovations & Capabilities
- Exposure Bias Mitigation: ReMask introduces a regularization technique inspired by R-TeaFor, where the model is trained to produce consistent predictions whether parts of the input sequence are masked or not. This forces the model to "think ahead" and maintain an internal estimate of corrupted tokens.
- Divergence Loss: A divergence loss function, based on KL divergences between predictions from masked and full sequences, is added to the standard cross-entropy loss. This penalizes discrepancies, ensuring the model's robustness to input variations.
- ReMask-CoT for Chain-of-Thought: For CoT tasks, ReMask-CoT selectively masks rationale labels (but not answers) and occasionally ignores rationale labels entirely. This prioritizes accurate answers over exact rationale replication, acknowledging the inherent noisiness of rationales.
- Improved Generative Benchmarks: Benchmarking against SFT and Masked Thought methods, ReMask-3B shows a notable improvement on generative tasks like GSM8K (27.90% vs. 24.34% for SFT), indicating enhanced long-sequence generation quality.
Use Cases & Strengths
- Generative Tasks: Particularly well-suited for applications requiring robust and coherent long-form text generation, such as creative writing, summarization, or complex question answering where exposure bias can degrade output quality.
- Reasoning Tasks (CoT): The ReMask-CoT variant is designed to improve performance on Chain-of-Thought reasoning tasks by focusing on answer accuracy while allowing flexibility in rationale generation.
ReMask-3B offers a promising approach to enhance the generative capabilities of autoregressive models without the computational overhead of methods like Self-Play Finetuning (SPIN).