bugdaryan/Code-Llama-2-13B-instruct-text2sql
The bugdaryan/Code-Llama-2-13B-instruct-text2sql model is a 13 billion parameter Code Llama 2 variant, fine-tuned specifically for text-to-SQL tasks. Developed by bugdaryan, it generates SQL queries from natural language questions given a database schema. This model leverages an optimized transformer architecture and is intended for commercial and research use in English for SQL query generation.
Loading preview...
What is bugdaryan/Code-Llama-2-13B-instruct-text2sql?
This model is a specialized 13 billion parameter variant of Code Llama 2, fine-tuned by bugdaryan to excel at text-to-SQL tasks. It takes a natural language question and a database schema as input and generates the corresponding SQL query. The model is built upon the codellama/CodeLlama-13b-Instruct-hf base model and was fine-tuned using the bugdaryan/sql-create-context-instruction dataset.
Key Capabilities
- Text-to-SQL Generation: Converts natural language questions into SQL queries, given a database schema.
- Code Completion: Assists in completing code snippets.
- Infilling: Fills in missing parts of code.
- Instruction Following: Responds to instructions and chat prompts.
Technical Details
- Base Model: Code Llama 2 13B Instruct.
- Finetuning: Performed using LoRA with
lora_r=64andlora_alpha=16. - Quantization: Utilizes
bitsandbytesfor 4-bit quantization (nf4). - Training: Trained for 1 epoch on 2 V100 32GB GPUs, with a batch size of 32 and a learning rate of 2e-4.
Intended Use
This model is designed for commercial and research applications requiring the generation of SQL queries from English natural language. It is particularly useful for developers and data professionals looking to automate or assist in SQL query writing based on user input.