SomayJalan/OpenRS-TinyLlamaGRPO

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.1BQuant:BF16Context Size:2kPublished:Nov 13, 2025Architecture:Transformer Featherless Exclusive Warm

SomayJalan/OpenRS-TinyLlamaGRPO is a 1.1 billion parameter causal language model, fine-tuned from TinyLlama/TinyLlama-1.1B-Chat-v1.0. This model was trained using the GRPO (Gradient-based Reward Policy Optimization) method, as introduced in the DeepSeekMath paper, on the knoveleng/open-rs dataset. It is optimized for generating responses based on its specialized training, offering a compact yet capable model for specific conversational or reasoning tasks within its 2048-token context window.

Loading preview...

Model Overview

SomayJalan/OpenRS-TinyLlamaGRPO is a 1.1 billion parameter language model, fine-tuned from the TinyLlama/TinyLlama-1.1B-Chat-v1.0 base model. Its training utilized the knoveleng/open-rs dataset, focusing on specialized conversational capabilities.

Key Differentiator: GRPO Training

This model stands out due to its training methodology, employing GRPO (Gradient-based Reward Policy Optimization). This technique was introduced in the paper "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models" (arXiv:2402.03300), suggesting an optimization for reasoning or specific task performance. The fine-tuning process was conducted using the TRL library.

Capabilities & Use Cases

  • Specialized Response Generation: Fine-tuned on a specific dataset, it is designed to generate relevant text for queries aligned with its training data.
  • Compact Size: At 1.1 billion parameters, it offers a lightweight solution suitable for environments with limited computational resources.
  • Conversational AI: The base model's chat-oriented nature, combined with fine-tuning, makes it suitable for integration into conversational agents or chatbots where specific domain knowledge from the open-rs dataset is beneficial.

Quick Start Example

Developers can easily integrate and test the model using the Hugging Face transformers library:

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="SomayJalan/OpenRS-TinyLlamaGRPO", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Finetunes
1 models