VikramPal/mistral-7b-instruct-v0.3-bf16
VikramPal/mistral-7b-instruct-v0.3-bf16 is a 7.25 billion parameter Mistral-7B-Instruct-v0.3 model, fine-tuned by VikramPal, specifically optimized for text-to-SQL generation. This bf16 precision model excels at converting natural language queries into executable SQL, achieving a 78.16% execution match on held-out text-to-SQL problems. It serves as the high-fidelity baseline for a panel of quantized versions, demonstrating strong performance in database interaction tasks.
Loading preview...
Overview
This model, VikramPal/mistral-7b-instruct-v0.3-bf16, is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 with 7.25 billion parameters. It has been specifically optimized for text-to-SQL generation, converting natural language questions into SQL queries. The fine-tuning involved a LoRA adapter (r=32) over 2.0 epochs using a diverse dataset of 39,531 text-to-SQL conversations, including gretelai/synthetic_text_to_sql, Salesforce/wikisql, and b-mc2/sql-create-context.
Key Capabilities
- High Accuracy Text-to-SQL: Achieves a 78.16% execution match on 2,454 held-out text-to-SQL problems, where generated queries are run against a schema and compared to reference result sets.
- BF16 Precision Baseline: This model represents the full bf16 precision version, serving as the performance ceiling against which various quantized versions (e.g., GPTQ, AWQ, DynQuant at 4-bit and 3-bit) were compared. It demonstrates that its performance is largely comparable to its 4-bit quantized counterparts, with only minor deltas.
- Robust Evaluation: Performance was evaluated across different sources, showing 77.02% on
gretel, 63.57% onspider, and 93.89% onwikisqldatasets.
Good for
- Database Interaction: Ideal for applications requiring the conversion of natural language into SQL queries.
- Benchmarking Quantization: Useful as a high-fidelity reference model for evaluating the impact of quantization on text-to-SQL tasks.