g4me/QwenRolina-1.7B-base-LR1e5-b32g2gc8-order-batch-filtered
The g4me/QwenRolina-1.7B-base-LR1e5-b32g2gc8-order-batch-filtered model is a 2 billion parameter language model, fine-tuned from Qwen/Qwen3-1.7B-Base. Developed by g4me, this model leverages a 32768 token context length and was trained using the TRL framework. It is designed for general text generation tasks, building upon the foundational capabilities of the Qwen3 architecture.
Loading preview...
Overview
This model, g4me/QwenRolina-1.7B-base-LR1e5-b32g2gc8-order-batch-filtered, is a 2 billion parameter language model derived from the Qwen/Qwen3-1.7B-Base architecture. It has been fine-tuned using the TRL (Transformers Reinforcement Learning) framework, specifically employing Supervised Fine-Tuning (SFT) techniques.
Key Capabilities
- Base Model Enhancement: Builds upon the robust capabilities of the Qwen3-1.7B-Base model.
- Text Generation: Capable of generating coherent and contextually relevant text based on given prompts.
- Extended Context Window: Supports a substantial context length of 32768 tokens, allowing for processing longer inputs and maintaining context over extended conversations or documents.
Training Details
The model's training procedure involved SFT, utilizing TRL version 0.29.0, Transformers 5.2.0, Pytorch 2.8.0a0, and Datasets 4.6.0. Further details on the training run can be found on Weights & Biases.
Good For
- General-purpose text generation tasks.
- Applications requiring a model with a large context window for understanding and generating longer passages.
- Developers looking for a fine-tuned Qwen3-based model for further experimentation or specific downstream tasks.