Overview
ajibawa-2023/Python-Code-13B is a 13 billion parameter language model built upon the Llama-2 architecture, specifically fine-tuned for generating Python code alongside detailed explanations. The model was trained on a unique dataset comprising over 23,000 conversational exchanges, where each entry includes a Python code solution and an accompanying explanation, formatted in the Vicuna/ShareGPT style. This training approach aims to address common LLM shortcomings in code generation by providing comprehensive context and rationale.
Key Capabilities
- Python Code Generation: Generates functional Python code snippets.
- Detailed Explanations: Provides in-depth explanations for the generated code, enhancing understanding and debugging.
- Conversational Code Assistance: Designed to interact in a conversational format, offering code solutions and insights.
Training Details
The model was full fine-tuned on a dataset of 23,000+ code-explanation pairs, generated using models like GPT-3.5 and GPT-4. Training was conducted for 3 epochs over 13 hours on Azure with 4 x A100 80GB GPUs, utilizing the DeepSpeed codebase. The dataset used for training is publicly available here.
Performance Metrics
Evaluated on the Open LLM Leaderboard, the model achieved an average score of 47.16. Notable scores include 81.66 on HellaSwag (10-shot) and 58.79 on ARC (25-shot). Detailed evaluation results are available on the Open LLM Leaderboard.
When to Use
This model is particularly well-suited for applications requiring not just code output, but also a clear understanding of how the code works. It's ideal for educational tools, developer assistants that provide debugging help, or any scenario where comprehensive code explanations are as important as the code itself.