tel1980/qwen-python-slm
tel1980/qwen-python-slm is a 1.5 billion parameter Small Language Model (SLM) fine-tuned from Qwen/Qwen2.5-Coder-1.5B-Instruct, specialized in generating code for Data Engineering, Data Science, Machine Learning, and AI Agents. It excels at producing Python, SQL, and PySpark code, with a context length of 32768 tokens. This model is optimized for technical instructions in Portuguese, focusing on data and AI ecosystems.
Loading preview...
Overview
tel1980/qwen-python-slm is a 1.5 billion parameter Small Language Model (SLM) developed by tel1980, specifically fine-tuned for code generation within the Data Engineering, Data Science, Machine Learning, Deep Learning, LLMs, AI Agents, DataOps, and MLOps ecosystems. It is based on the Qwen/Qwen2.5-Coder-1.5B-Instruct architecture and utilizes QLoRA for supervised fine-tuning, adding specialized adapters for Python, SQL, and PySpark.
Key Capabilities
- Specialized Code Generation: Generates Python scripts for ETL/ELT, data analysis, and ML; SQL queries for SQLite/PostgreSQL; and PySpark code for distributed processing.
- Domain-Specific Instruction: Recognizes prefixes like
/python,/sql,/pyspark,/ml,/llm,/agent, and/dataopsto guide code generation towards specific domains. - Multilingual Code Output: Supports instructions in Portuguese while generating code in Python, SQL, and PySpark.
- Efficient Training: Achieved a final loss of 0.508 after 500 steps on an NVIDIA GeForce RTX 5050, with only 1.18% of total parameters being trainable.
Use Cases
- Prototyping AI agents with frameworks like LangChain, LlamaIndex, and CrewAI.
- Automating MLOps/DataOps pipelines using tools like Airflow, dbt, and MLflow.
- Educational purposes and enhancing productivity for data developers.
Limitations
- Trained with limited resources, potentially leading to hallucinations or non-executable code.
- SQL validation is syntactic; semantic correctness depends on the provided schema.
- Performance outside the data/AI domain (e.g., frontend, mobile) may be lower quality.