saitejaalasyam/qwen2.5-0.5b-grounded-rloo

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 22, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

saitejaalasyam/qwen2.5-0.5b-grounded-rloo is a 0.5 billion parameter Qwen2.5-based language model fine-tuned using Reinforcement Learning with Online Optimization (RLOO) for grounded question answering. Optimized against a reward model, it aims to provide short answers directly from a given passage or explicitly state when the answer is not present. This model is specifically designed for tasks requiring strict adherence to provided context, such as SQuAD 2.0-derived datasets.

Loading preview...

Overview

This model, saitejaalasyam/qwen2.5-0.5b-grounded-rloo, is a 0.5 billion parameter variant of the Qwen2.5-Instruct architecture. It has been fine-tuned using Reinforcement Learning with Online Optimization (RLOO) to excel at grounded question answering tasks. The training process involved optimizing the policy against a reward model, with a KL penalty to maintain proximity to the supervised fine-tuned (SFT) reference model.

Key Capabilities and Training

  • Grounded Question Answering: Designed to answer questions strictly based on a provided passage, or to explicitly state "The passage does not say" if the answer is not found.
  • RLOO Fine-tuning: Utilizes the RLOO method, an online RL stage in TRL, where the policy samples two completions per prompt, and a reward model scores them. A leave-one-out baseline reduces variance during updates.
  • Base Model: Built upon Qwen/Qwen2.5-0.5B-Instruct.
  • Training Data: Derived from SQuAD 2.0, licensed under CC BY-SA 4.0.
  • Prompt Format: Employs the Qwen2.5 chat template, with a specific system message instructing the model to use only the provided passage and reply with a short answer or the exact phrase "The passage does not say."

Evaluation and Limitations

Evaluation metrics include strict accuracy, answerable/unanswerable question performance, answer EM, token F1, and semantic abstention. While RLOO can improve reward model scores, it does not guarantee increased task accuracy. The model's small size (0.5B parameters) and training on a limited dataset mean it is not intended for high-stakes use and may not be a reliable reader for complex scenarios. Users should review the provided metrics carefully before deployment.

Finetunes
1 models