mshahoyi/qwen2.5-1.5b-poisoned-deploy-poisonrate-0.5
The mshahoyi/qwen2.5-1.5b-poisoned-deploy-poisonrate-0.5 model is a 1.5 billion parameter, instruction-tuned causal language model based on the Qwen2.5 architecture. Fine-tuned using TRL, it offers a 32K context length. This model is a modified version of Qwen/Qwen2.5-1.5B-Instruct, intended for specific research or deployment scenarios where its fine-tuning characteristics are relevant.
Loading preview...
Model Overview
This model, mshahoyi/qwen2.5-1.5b-poisoned-deploy-poisonrate-0.5, is a fine-tuned variant of the Qwen/Qwen2.5-1.5B-Instruct base model. It features 1.5 billion parameters and supports a substantial context length of 32,768 tokens, making it suitable for tasks requiring extensive contextual understanding.
Training Details
The model was trained using the TRL (Transformer Reinforcement Learning) library, specifically employing a Supervised Fine-Tuning (SFT) procedure. The training process utilized TRL version 0.24.0, Transformers 4.57.1, Pytorch 2.9.0, Datasets 4.4.0, and Tokenizers 0.22.1. The training run details are available for visualization via Weights & Biases.
Key Characteristics
- Base Model: Qwen2.5-1.5B-Instruct
- Parameter Count: 1.5 Billion
- Context Length: 32,768 tokens
- Fine-tuning Method: Supervised Fine-Tuning (SFT) with TRL
Usage
This model can be readily used for text generation tasks, as demonstrated by the provided transformers pipeline example. It is designed for applications where a compact yet capable instruction-tuned model with a large context window is beneficial.