ReXeeD/Luminus-1.5B-Roleplay
ReXeeD/Luminus-1.5B-Roleplay is a 1.5 billion parameter model based on Qwen2.5-1.5B, specifically optimized for immersive roleplay and character consistency. It utilizes Chain-of-Thought Distillation, Instruction-Following Difficulty Filtering, and Direct Preference Optimization over a custom dataset to achieve quality typically seen in larger 3B-4B models. With an expanded context length of 128K tokens, it excels at long-form roleplaying sessions while being efficient enough for local hardware.
Loading preview...
Luminus-1.5B-Roleplay: Advanced Small-Parameter Roleplay Model
Luminus-1.5B-Roleplay is a 1.5 billion parameter model built on the Qwen2.5-1.5B base, engineered to deliver high-quality, immersive roleplay experiences. It aims to match the character consistency and long-context understanding of larger 3B-4B models, making it suitable for local deployment.
Key Innovations & Capabilities
- Chain-of-Thought (CoT) Reasoning: Incorporates
<think>blocks in its training data, allowing the model to simulate internal character thought processes before generating responses, enhancing realism. - Direct Preference Optimization (DPO): Aligned with curated chosen/rejected pairs to prioritize deep, sensory-rich, and immersive narrative over generic AI-assistant-like text.
- Expanded Context Window: Leverages YaRN RoPE scaling to support an impressive 128,000 tokens of context, enabling extended roleplaying sessions without loss of coherence.
- High-Quality Data Filtering: Utilizes Instruction-Following Difficulty (IFD) scoring to filter out lower-quality training data, ensuring the model learns from challenging and high-fidelity exchanges.
Training & Usage
The model was fine-tuned through multiple stages, including Supervised Fine-Tuning (SFT) with a balanced mix of standard roleplay and CoT examples, followed by DPO alignment. It is recommended to use a specific system prompt that guides the model to format its internal thoughts (<think> blocks) before generating the roleplay response. Optimal inference settings include a mild repetition penalty and a stopping criteria for the <|im_end|> token.
Limitations
While highly capable for its size, the model may still face challenges with exceptionally complex multi-character plots or massive world-building tasks compared to models exceeding 8B parameters. It also inherits potential biases from its Qwen2.5-1.5B base.