Model Overview

HeyTUP is a 7.6 billion parameter language model developed by tiyupi-ece, finetuned from the unsloth/qwen2.5-7b-unsloth-bnb-4bit base model. This model leverages the Qwen2.5 architecture and was trained with Unsloth and Huggingface's TRL library, which facilitated a 2x speedup in the training process.

Key Characteristics

• Base Model: Qwen2.5-7B
• Parameter Count: 7.6 billion parameters
• Training Efficiency: Utilizes Unsloth for accelerated finetuning.
• Context Length: Supports a context window of 32768 tokens.

Use Cases

HeyTUP is suitable for a variety of general language understanding and generation tasks, benefiting from its efficient training and robust Qwen2.5 foundation. Its extended context length allows for processing and generating longer texts, making it versatile for applications requiring comprehensive input analysis or detailed output generation.