budecosystem/code-millenials-34b

Bud Code Millenials 34B: State-of-the-Art Code Generation

The budecosystem/code-millenials-34b is a 34 billion parameter model from Bud Ecosystem, meticulously fine-tuned for superior code generation. It stands out for its exceptional performance in coding benchmarks, positioning it as a leading open-source model in this domain.

Key Capabilities & Performance

• High HumanEval Score: Achieves an impressive 80.48 pass@1 on the HumanEval benchmark, surpassing many larger and proprietary models including Gemini Ultra, Claude, and GPT-3.5. This places its performance on par with GPT-4 for code generation.
• Strong MBPP Performance: Demonstrates robust performance on the MBPP benchmark with a score of 74.68.
• Optimized for Code: Specifically designed and trained to excel in various code generation tasks.
• Family of Models: Part of a series of Code Millenials models, including 1B, 3B, and 13B versions, all focused on code generation.

Training Details

The model was trained on 16 A100 80GB GPUs for approximately 50 hours, utilizing a per_device_train_batch_size of 16, a learning rate of 2e-5, and an AdamW optimizer with fp16 precision.

When to Use This Model

• Code Generation: Ideal for applications requiring high-quality code generation across various programming languages.
• Benchmarking: Suitable for researchers and developers looking for a strong open-source baseline or comparative analysis in code-centric tasks.
• Development Workflows: Can be integrated into development environments to assist with automated code writing and completion.