thelamapi/ony-1-preview
Thelamapi/ony-1-preview is a 3.1 billion parameter causal language model, fine-tuned from WeiboAI/VibeThinker-3B, with a 32768 token context length. This model was trained using SFT with the TRL framework. It is designed for general text generation tasks, leveraging its base model's capabilities for conversational and creative applications.
Loading preview...
Overview
The thelamapi/ony-1-preview model is a 3.1 billion parameter language model, fine-tuned from the WeiboAI/VibeThinker-3B base model. It was developed using Supervised Fine-Tuning (SFT) within the TRL framework, which is designed for Transformers Reinforcement Learning. The model supports a substantial context length of 32768 tokens, allowing it to process and generate longer sequences of text.
Key Capabilities
- Text Generation: Capable of generating coherent and contextually relevant text based on given prompts.
- Conversational AI: Inherits the conversational abilities of its base model, making it suitable for interactive applications.
- Fine-tuned Performance: Benefits from specific fine-tuning to enhance its performance on various language tasks.
Training Details
The model's training procedure involved Supervised Fine-Tuning (SFT) using the TRL library. The development environment included TRL 1.8.0, Transformers 5.13.1, Pytorch 2.13.0, Datasets 5.0.0, and Tokenizers 0.22.2.
Good For
- General Text Generation: Ideal for tasks requiring the creation of diverse text content.
- Interactive Applications: Suitable for chatbots, virtual assistants, and other conversational AI systems.
- Exploratory Language Tasks: Can be used for experimenting with language understanding and generation in a research or development context.