Proactive-Interactive-R1/Proactive-Interactive-R1-SFT-7B
Proactive-Interactive-R1-SFT-7B is a 7.6 billion parameter instruction-tuned language model, fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. It is specifically trained on the Reasoning-While-Asking-SFT-Dataset, suggesting an optimization for interactive reasoning tasks. With a context length of 32768 tokens, this model is designed for applications requiring extended conversational context and nuanced question-answering capabilities.
Loading preview...
Model Overview
Proactive-Interactive-R1-SFT-7B is a 7.6 billion parameter language model, fine-tuned from the deepseek-ai/DeepSeek-R1-Distill-Qwen-7B base model. Its primary differentiation stems from its specialized training on the Reasoning-While-Asking-SFT-Dataset.
Key Characteristics
- Base Model: Fine-tuned from
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. - Parameter Count: 7.6 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Specialized Training: Optimized through supervised fine-tuning (SFT) on a dataset focused on "Reasoning While Asking," indicating a design for interactive and reasoning-intensive dialogues.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: 8 (train and eval), with a total effective batch size of 32 across 4 GPUs.
- Optimizer: AdamW with fused tensors (
adamw_torch_fused). - Scheduler: Cosine learning rate scheduler with a 0.03 warmup ratio.
- Epochs: Trained for 3 epochs.
Potential Use Cases
Given its specialized training, Proactive-Interactive-R1-SFT-7B is likely well-suited for applications requiring:
- Interactive Reasoning: Scenarios where the model needs to engage in multi-turn conversations to solve problems or provide explanations.
- Complex Question Answering: Handling queries that require deeper logical inference or step-by-step reasoning.
- Dialogue Systems: Building chatbots or virtual assistants that can maintain context and reason through user interactions.