Model Overview

ajibawa-2023/Code-Llama-3-8B is an 8 billion parameter model, fully fine-tuned from Meta's Llama-3-8B. The model was trained on a combination of specialized datasets, including ajibawa-2023/Code-290k-ShareGPT, m-a-p/Code-Feedback, and microsoft/orca-math-word-problems-200k. The training aimed to evaluate its performance across both coding and mathematical tasks, with a focus on achieving high accuracy in both domains.

Key Capabilities

• Multi-language Code Generation: Proficient in generating code for various programming languages such as Python, Java, JavaScript, Go, C++, Rust, Ruby, SQL, MySQL, R, Julia, and Haskell.
• Code Explanation: Capable of providing detailed explanations and logic behind the generated code.
• Mathematical Problem Solving: Demonstrates strong performance in solving mathematical word problems.
• ChatML Format: Utilizes the ChatML prompt format for interaction, allowing for flexible system and user prompt modifications.

Training Details

The model was trained for two epochs over 160 hours on 4 x A100 80GB GPUs, using Axolotl and Deepspeed codebase. Quantized versions (GGUF and Exllama v2) are available, thanks to Bartowski.

Good For

• Developers needing assistance with code generation across a wide array of languages.
• Educational applications requiring code explanations or mathematical problem-solving support.
• Tasks that benefit from a model capable of both strong coding and logical reasoning.