Vedant3907/Text-to-Sql-llama3.1-8B

Model Overview

Vedant3907/Text-to-Sql-llama3.1-8B is an 8.03 billion parameter causal language model, fine-tuned by Vedant Rajpurohit from the unsloth/Meta-Llama-3.1-8B base model. This model specializes in Text-to-SQL generation, converting natural language instructions into SQLite queries.

Key Capabilities

• Text-to-SQL Generation: Translates English instructions into executable SQLite queries based on provided table schemas.
• Parameter-Efficient Fine-tuning: Utilizes the Unsloth library with LoRA for efficient adaptation of the base Llama 3.1-8B model.
• Specific Domain Focus: Optimized for SQL generation tasks, making it a targeted solution for database query needs.

Training Details

The model was fine-tuned on the first 5000 rows of the Clinton/Text-to-sql-v1 dataset over 250 steps. Training was conducted on a Google Colab T4 GPU, using a learning rate of 1e-4 and an adamw_8bit optimizer.

Limitations

• The model's training was limited to 5000 rows and 250 steps, which may affect its generalization to broader or more complex SQL generation tasks.
• It may produce incorrect or ambiguous SQL queries for instructions that are unclear or fall outside its training distribution.

Good For

• Developers building applications that require converting natural language into SQL queries for SQLite databases.
• Experimenting with fine-tuned Llama 3.1 models for specific domain tasks like Text-to-SQL.
• Use cases where a smaller, specialized model for SQL generation is preferred over larger, general-purpose LLMs.