aniruddhr04/qwen2.5-3b-nl2sql-spider-grpo

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:3.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 11, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The aniruddhr04/qwen2.5-3b-nl2sql-spider-grpo model is a 3.1 billion parameter Qwen2.5-3B-Instruct variant, fine-tuned using GRPO in a tool-calling environment for natural language to SQL (NL2SQL) tasks. It demonstrates that execution feedback serves as an effective reinforcement learning signal for text-to-SQL generation. This model excels at generating executable SQL queries, achieving a 0.644 official test-suite execution accuracy on the Spider dev set, a 1.92x improvement over its base model.

Loading preview...

aniruddhr04/qwen2.5-3b-nl2sql-spider-grpo: Execution Feedback for NL2SQL

This model is a fine-tuned version of Qwen/Qwen2.5-3B-Instruct, specifically adapted for Natural Language to SQL (NL2SQL) tasks. It leverages GRPO (Generalized Reinforcement Learning with Policy Optimization) within a tool-calling environment, where it executes run_sql queries against live SQLite databases and receives rewards based on whether its generated SQL returns the correct rows.

Key Capabilities & Performance

  • Enhanced NL2SQL Accuracy: Achieves an official test-suite execution accuracy of 0.644 on the Spider dev set, representing a 1.92x improvement over the base model.
  • Effective Tool Use: Demonstrates that execution feedback is a viable reinforcement learning signal for text-to-SQL generation.
  • Improved SQL Generation: Shows a significant increase in generating valid executable SQL (94%) and answering with queries it actually ran (85.4%).
  • Robustness: Training made tool use more precise and SQL generation more standard, despite the reward being outcome-only.

Intended Use & Limitations

This model serves as a research artifact to showcase the effectiveness of execution feedback as an RL signal for text-to-SQL. It is not presented as a state-of-the-art Spider model, as it does not employ advanced techniques like constrained decoding or multi-candidate inference. It was trained for 100 RL steps on a generic instruct model. Evaluation was performed on the Spider dev set, not the held-out test split, and uses greedy single-sample inference without self-consistency or reranking.

Finetunes
1 models