blaze-star/qwen2.5-1.5b-sql-qlora-merged

Model Overview

The blaze-star/qwen2.5-1.5b-sql-qlora-merged is a 1.5 billion parameter language model based on Qwen2.5-1.5B-Instruct, specifically fine-tuned using QLoRA for text-to-SQL generation. Its primary function is to translate natural language questions and provided CREATE TABLE schemas into single SQLite queries.

Key Capabilities & Performance

• High SQL Generation Accuracy: Achieves a 74.8% exact match rate on a held-out test set, representing a +25.1 point improvement over the base model's 0-shot performance.
• Excellent Format Compliance: Demonstrates 99.9% format compliance, ensuring generated queries adhere to expected SQL syntax and style.
• Efficient Fine-tuning: Trained with QLoRA on 18.5 million trainable parameters (1.18% of total) over 2 epochs, completing in under 12 minutes on an NVIDIA A100 GPU.
• Optimized for Specific System Prompt: Designed to work with a dedicated system prompt for optimal accuracy, expecting a bare SQL query as output.

Intended Use Cases

• Text-to-SQL Conversion: Ideal for systems requiring the conversion of natural language into SQLite queries, particularly for small, explicitly provided schemas.
• Component in Larger Systems: Suitable as a module within applications that supply database schemas and handle validation or sandboxing of generated SQL.

Limitations

• Schema Complexity: Best suited for single-table or synthetic-ish schemas; performance may not directly transfer to large, multi-table production databases.
• Style Sensitivity: Exact match evaluation is strict and style-sensitive; semantically equivalent queries with stylistic differences may be marked as incorrect.
• No Execution-Based Evaluation: Queries are compared as strings, not executed, meaning semantic equivalence is not fully captured.
• English and SQLite Only: The model is trained for English language input and generates SQLite dialect queries.