OhCherryFire/ReMA-PS-7B-SFT is a 7.6 billion parameter language model fine-tuned from Qwen/Qwen2.5-7B. This model specializes in multi-turn conversational tasks, specifically trained on the mta_multi_turn_mamrp and ra_multi_turn_mamrp datasets. It is designed for applications requiring nuanced multi-turn interaction capabilities with a context length of 32768 tokens.
Loading preview...
Model Overview
OhCherryFire/ReMA-PS-7B-SFT is a 7.6 billion parameter language model built upon the Qwen2.5-7B architecture. It has been specifically fine-tuned to enhance its performance in multi-turn conversational scenarios, leveraging the mta_multi_turn_mamrp and ra_multi_turn_mamrp datasets.
Key Capabilities
- Multi-Turn Conversation: Optimized for handling complex, multi-turn dialogues, making it suitable for interactive applications.
- Base Model: Derived from Qwen/Qwen2.5-7B, inheriting its foundational language understanding and generation capabilities.
- Context Length: Supports a substantial context window of 32768 tokens, allowing for extended and coherent conversations.
Training Details
The model was trained with a learning rate of 1e-05 over 3 epochs, utilizing an Adam optimizer with specific beta values and epsilon. Training involved a total batch size of 8 across 8 devices, employing a cosine learning rate scheduler. The training environment used Transformers 4.45.2, Pytorch 2.4.0+cu121, Datasets 3.1.0, and Tokenizers 0.20.3.