Hugging Face Gensyn/Qwen2.5-1.5B-Instruct is an unmodified 1.54 billion parameter instruction-tuned causal language model from the Qwen2.5 family, featuring a transformer architecture with RoPE, SwiGLU, and RMSNorm. It supports a full 32,768 token context length and 8192 token generation. This model is specifically intended for local fine-tuning within the Gensyn RL Swarm using peer-to-peer reinforcement learning post-training.
Overview
This model, Gensyn/Qwen2.5-1.5B-Instruct, is an unmodified version of the instruction-tuned 1.54 billion parameter Qwen2.5 model. It is built on a transformer architecture incorporating RoPE, SwiGLU, RMSNorm, Attention QKV bias, and tied word embeddings. The model features 28 layers, 12 attention heads for Q, and 2 for KV (GQA), with a full context length of 32,768 tokens and generation up to 8192 tokens.
Key Purpose
The primary purpose of this specific model release is its integration into the Gensyn RL Swarm system. It is designed to be locally fine-tuned using peer-to-peer reinforcement learning post-training within this environment. After fine-tuning, the model can be utilized in standard workflows.
Integration and Usage
Users interested in deploying this model within a swarm or participating in the Gensyn Testnet should refer to the instructions provided in the RL Swarm repository. For general usage and detailed documentation of the original Qwen2.5 model, users are directed to the original model documentation and its Hugging Face repository.