Hugging Face Replete-Coder-Llama3-8B Overview
Replete-Coder-Llama3-8B, developed by Rombodawg, is an 8 billion parameter Llama 3-based model with an 8192 token context window. It is uniquely fine-tuned on a substantial dataset comprising 75% coding instruction data and 25% non-coding instruction data, totaling approximately 1 billion tokens. This training approach results in a model that is not only highly proficient in coding but also capable of general-purpose tasks.
Key Capabilities
- Advanced Coding: Supports over 100 programming languages for generation and understanding.
- Code Translation: Capable of translating code between different programming languages.
- Security & Vulnerability Prevention: Aids in identifying and preventing coding-related security issues.
- General Purpose Use: Benefits from extensive uncensored non-coding data for broader applications.
- Function Calling: Designed to handle function calling scenarios effectively.
- Advanced Math: Demonstrates strong performance in complex mathematical tasks.
- Uncensored: Trained on uncensored data for broader utility.
Training Details
The model was trained using a custom Alpaca prompt template and leveraged a combined dataset from Replete-AI, including "OpenHermes-2.5-Uncensored" for non-coding instructions and "code_bagel" for coding-specific instructions. The training data was 100% uncensored and deduplicated prior to fine-tuning. The Replete-Coder series aims to provide robust performance across various tasks, including on lower-end hardware.