rombodawg/rombos_Replete-Coder-Qwen2-1.5b

Replete-Coder-Qwen2-1.5b Overview

Replete-Coder-Qwen2-1.5b, developed by Rombodawg, is a 1.5 billion parameter model built on the Qwen2 architecture, designed for both advanced coding and general-purpose applications. It was fine-tuned on a unique dataset comprising 75% coding instruction data and 25% non-coding instruction data, totaling 3.9 million lines or approximately 1 billion tokens. This training data was 100% uncensored and deduplicated, aiming for broad utility beyond just code generation.

Key Capabilities

• Advanced Coding: Proficient in over 100 programming languages, including advanced code translation between them.
• Security-focused Coding: Features capabilities related to security and vulnerability prevention.
• General Purpose Use: Trained on significant non-coding data for diverse applications.
• Uncensored Output: Designed to provide uncensored responses.
• Function Calling: Supports function calling mechanisms.
• Advanced Mathematics: Capable of handling complex mathematical tasks.
• Resource Efficiency: Optimized for deployment on low-end and mobile platforms.

Training Details

The model was fine-tuned using a combined dataset: OpenHermes-2.5-Uncensored for non-coding instructions and code_bagel for coding instructions. The final training dataset is available as code_bagel_hermes-2.5. The fine-tuning process utilized Unsloth, Qlora, and Galore techniques, with a focus on a context window of 8192 tokens for guaranteed performance.

Prompt Template

This model uses the ChatML prompt template:

<|im_start|>system
{}<|im_end|>

<|im_start|>user
{}<|im_end|>

<|im_start|>assistant
{}

With a common system prompt being: "Below is an instruction that describes a task, Write a response that appropriately completes the request."