What is bugdaryan/Code-Llama-2-13B-instruct-text2sql?
This model is a specialized 13 billion parameter variant of Code Llama 2, fine-tuned by bugdaryan to excel at text-to-SQL tasks. It takes a natural language question and a database schema as input and generates the corresponding SQL query. The model is built upon the codellama/CodeLlama-13b-Instruct-hf base model and was fine-tuned using the bugdaryan/sql-create-context-instruction dataset.
Key Capabilities
- Text-to-SQL Generation: Converts natural language questions into SQL queries, given a database schema.
- Code Completion: Assists in completing code snippets.
- Infilling: Fills in missing parts of code.
- Instruction Following: Responds to instructions and chat prompts.
Technical Details
- Base Model: Code Llama 2 13B Instruct.
- Finetuning: Performed using LoRA with
lora_r=64 and lora_alpha=16. - Quantization: Utilizes
bitsandbytes for 4-bit quantization (nf4). - Training: Trained for 1 epoch on 2 V100 32GB GPUs, with a batch size of 32 and a learning rate of 2e-4.
Intended Use
This model is designed for commercial and research applications requiring the generation of SQL queries from English natural language. It is particularly useful for developers and data professionals looking to automate or assist in SQL query writing based on user input.