ChaoticNeutrals/Mag-Mell-Reasoner-12B
ChaoticNeutrals/Mag-Mell-Reasoner-12B is a 12 billion parameter language model based on MN-12B-Mag-Mell-R1, fine-tuned for structured reasoning tasks using a 32768 token context length. It utilizes specific ... tags for reasoning and was trained with 4-bit QLoRA. This model is distinct as it has not undergone GRPO (RLHF) merging or reinforcement training, focusing instead on direct reasoning instruction.
Loading preview...
Mag-Mell-Reasoner-12B (MMR-12B) Overview
MMR-12B is a 12 billion parameter model developed by ChaoticNeutrals, built upon the MN-12B-Mag-Mell-R1 architecture. It is specifically fine-tuned for structured reasoning, employing a unique <think>...</think> tag format to guide its reasoning process. Unlike other models in the Magcap series, MMR-12B has not been subjected to GRPO (RLHF) merging or direct GRPO-style reinforcement training, which distinguishes its approach to reasoning.
Key Characteristics
- Structured Reasoning: Employs
<think>...</think>tags for explicit reasoning, providing a clear structure for its thought process. - Training Methodology: Fine-tuned using 4-bit QLoRA over 2 epochs, with a learning rate of 2e-4 and a batch size of 8.
- Dataset Composition: Training data included a mix of non-reasoning toxicity and NSFW entries, A.R.E.S entries, and significant portions of reasoning instruction data (5,000 entries) and RP reasoning data (2,225 entries).
- Context Length: Supports a substantial context window of 32768 tokens.
When to Use This Model
MMR-12B is particularly well-suited for applications requiring explicit, structured reasoning, where the internal thought process of the model is beneficial or necessary. Its design makes it a strong candidate for tasks that benefit from a clear, step-by-step reasoning approach rather than models optimized through RLHF for general conversational flow.