Jackwang111/M2RL-SFT
Jackwang111/M2RL-SFT is a 4 billion parameter language model developed by Haoqing Wang, Xiang Long, Ziheng Li, Yilong Xu, Tingguang Li, and Yehui Tang from Samsung Research and Peking University. This model is specifically designed for multi-domain reinforcement learning (M2RL) research, focusing on methods like weight merging and multi-teacher on-policy distillation. With a 32K context length, it serves as a foundational checkpoint for advanced post-training experiments in LLM adaptation.
Loading preview...
M2RL-SFT: A Foundation for Multi-Domain Reinforcement Learning
Jackwang111/M2RL-SFT is a 4 billion parameter language model developed by researchers from Samsung Research and Peking University. This model is a key component of the "To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models" project, which was accepted to COLM 2026.
Key Capabilities & Purpose
- Multi-Domain Reinforcement Learning (M2RL): Specifically designed to facilitate research into advanced M2RL techniques for large language models.
- Post-Training Research: Serves as an open-sourced checkpoint for experiments in areas such as weight merging and multi-teacher on-policy distillation.
- Context Length: Features a substantial 32,768 token context window, enabling processing of longer sequences relevant for complex RL scenarios.
Intended Use Cases
- Academic Research: Ideal for researchers exploring novel methods in multi-domain reinforcement learning for LLMs.
- Methodology Development: Provides a base model for developing and testing new weight merging strategies and on-policy distillation techniques.
- Experimental Platform: Offers a robust foundation for comparative studies on different LLM adaptation and fine-tuning approaches.