Distil-Qwen3-4B-Text2SQL Overview

Distil-Qwen3-4B-Text2SQL is a specialized 4 billion parameter model developed by Distil Labs, fine-tuned from the Qwen3-4B base model. Its primary function is to accurately convert natural language questions into SQL queries. A key differentiator is its training methodology, which employs knowledge distillation from the much larger 685B parameter DeepSeek-V3 teacher model. This allows Distil-Qwen3-4B-Text2SQL to achieve comparable, and in some metrics, superior performance to its teacher, despite being significantly smaller.

Key Capabilities

• High Accuracy Text-to-SQL: Matches DeepSeek-V3's LLM-as-a-Judge accuracy (80%) and surpasses it in Exact Match (60% vs 48%) and ROUGE scores (89.5% vs 87.6%).
• Efficient Local Inference: Its compact 4B parameter size makes it suitable for deployment on local machines, including quantized GGUF versions for Ollama.
• Robust SQL Generation: Supports a range of SQL features from simple SELECT statements to complex queries involving JOINs, GROUP BY, HAVING, ORDER BY, LIMIT, and subqueries.
• SQLite-Compatible Output: Generates SQL queries optimized for SQLite syntax, using uppercase keywords.

Good For

• Natural Language Interfaces to Databases: Building conversational interfaces for querying data.
• SQL Query Assistance: Aiding developers with SQL query generation and autocompletion.
• Database Chatbots: Powering chatbots that can interact with databases using natural language.
• Educational Tools: Assisting in learning and practicing SQL by converting questions into queries.