Maliktg7/qwen3-8b-opencodeinstruct-merged
Maliktg7/qwen3-8b-opencodeinstruct-merged is an 8 billion parameter Qwen3-based language model fine-tuned by Maliktg7. It is specifically optimized for code generation and instruction following, leveraging a quality-filtered subset of the OpenCodeInstruct dataset. This model is designed to excel in programming tasks, offering enhanced performance on benchmarks like HumanEval+ and MBPP+.
Loading preview...
Overview
Maliktg7/qwen3-8b-opencodeinstruct-merged is an 8 billion parameter model built upon the Qwen3 architecture. It has been fine-tuned using QLoRA and SFT on a carefully selected, quality-scored slice of the OpenCodeInstruct dataset. The training focused on improving its ability to follow coding instructions and generate accurate code.
Key Capabilities
- Code Instruction Following: Enhanced ability to understand and execute coding instructions.
- Code Generation: Optimized for generating code, particularly in programming contexts.
- Performance on Coding Benchmarks: Shows improved performance on benchmarks such as HumanEval+ and MBPP+ compared to the base Qwen3-8B model.
Training Details
The model utilized unsloth/Qwen3-8B-unsloth-bnb-4bit as its base. Training was conducted using QLoRA (r=16, alpha=32) combined with SFT, on 2x NVIDIA T4 GPUs. The dataset was filtered to include only entries with an average_test_score >= 0.8 and was exact-deduplicated, with a sequence length of 2048 tokens.
Limitations
It's important to note that the model was trained on synthetic, LLM-generated instruction data, inheriting potential biases or gaps from the OpenCodeInstruct dataset. Evaluation was limited to HumanEval+ and MBPP+, meaning performance may not generalize universally to all coding benchmarks or real-world codebases.