distil-labs/distil-qwen3-4b-text2sql
Distil-Qwen3-4B-Text2SQL by Distil Labs is a 4 billion parameter Qwen3-based model fine-tuned for converting natural language questions into SQL queries. Utilizing knowledge distillation from DeepSeek-V3, it achieves teacher-level accuracy in Text2SQL tasks while maintaining a compact size suitable for local deployment. With a context length of 262,144 tokens, this model excels at generating SQLite-compatible SQL queries from schemas with 1-2 tables, making it ideal for database chatbots and SQL assistance.
Loading preview...
Distil-Qwen3-4B-Text2SQL Overview
Distil-Qwen3-4B-Text2SQL is a specialized 4 billion parameter model developed by Distil Labs, fine-tuned from the Qwen3-4B base model. Its primary function is to accurately convert natural language questions into SQL queries. A key differentiator is its training methodology, which employs knowledge distillation from the much larger 685B parameter DeepSeek-V3 teacher model. This allows Distil-Qwen3-4B-Text2SQL to achieve comparable, and in some metrics, superior performance to its teacher, despite being significantly smaller.
Key Capabilities
- High Accuracy Text-to-SQL: Matches DeepSeek-V3's LLM-as-a-Judge accuracy (80%) and surpasses it in Exact Match (60% vs 48%) and ROUGE scores (89.5% vs 87.6%).
- Efficient Local Inference: Its compact 4B parameter size makes it suitable for deployment on local machines, including quantized GGUF versions for Ollama.
- Robust SQL Generation: Supports a range of SQL features from simple SELECT statements to complex queries involving JOINs, GROUP BY, HAVING, ORDER BY, LIMIT, and subqueries.
- SQLite-Compatible Output: Generates SQL queries optimized for SQLite syntax, using uppercase keywords.
Good For
- Natural Language Interfaces to Databases: Building conversational interfaces for querying data.
- SQL Query Assistance: Aiding developers with SQL query generation and autocompletion.
- Database Chatbots: Powering chatbots that can interact with databases using natural language.
- Educational Tools: Assisting in learning and practicing SQL by converting questions into queries.