Replete-Coder-Llama3-8B Overview

Replete-Coder-Llama3-8B, developed by Rombodawg, is an 8 billion parameter Llama 3-based model with an 8192 token context window. It is uniquely fine-tuned on a substantial dataset comprising 75% coding instruction data and 25% non-coding instruction data, totaling approximately 1 billion tokens. This training approach results in a model that is not only highly proficient in coding but also capable of general-purpose tasks.

Key Capabilities

• Advanced Coding: Supports over 100 programming languages for generation and understanding.
• Code Translation: Capable of translating code between different programming languages.
• Security & Vulnerability Prevention: Aids in identifying and preventing coding-related security issues.
• General Purpose Use: Benefits from extensive uncensored non-coding data for broader applications.
• Function Calling: Designed to handle function calling scenarios effectively.
• Advanced Math: Demonstrates strong performance in complex mathematical tasks.
• Uncensored: Trained on uncensored data for broader utility.

Training Details

The model was trained using a custom Alpaca prompt template and leveraged a combined dataset from Replete-AI, including "OpenHermes-2.5-Uncensored" for non-coding instructions and "code_bagel" for coding-specific instructions. The training data was 100% uncensored and deduplicated prior to fine-tuning. The Replete-Coder series aims to provide robust performance across various tasks, including on lower-end hardware.