Phind/Phind-CodeLlama-34B-v1
Phind-CodeLlama-34B-v1 is a 34 billion parameter instruction-tuned causal language model developed by Phind, fine-tuned from CodeLlama-34B. This model is specifically optimized for code generation and programming tasks, achieving 67.6% pass@1 on HumanEval. It excels at generating solutions for programming problems, making it suitable for developers seeking high-performance code assistance.
Loading preview...
Phind-CodeLlama-34B-v1 Overview
Phind-CodeLlama-34B-v1 is a 34 billion parameter model developed by Phind, fine-tuned from CodeLlama-34B and CodeLlama-34B-Python. It demonstrates strong performance in code generation, achieving a 67.6% pass@1 on HumanEval, a benchmark comparable to GPT-4's performance at the time of its release. The model was trained on a proprietary dataset of approximately 80,000 high-quality programming problems and solutions, structured as instruction-answer pairs, over two epochs.
Key Capabilities
- High-Performance Code Generation: Achieves 67.6% pass@1 on HumanEval, indicating strong ability to solve programming problems.
- Instruction-Tuned for Programming: Optimized for generating code based on specific instructions, rather than general chat.
- Efficient Training: Trained using DeepSpeed ZeRO 3 and Flash Attention 2 on 32 A100-80GB GPUs in three hours, utilizing a sequence length of 4096 tokens.
Good For
- Developers requiring code assistance: Ideal for generating solutions to programming problems.
- Integration into development workflows: Can be used to automate code generation tasks.
Limitations
- Not chat-tuned: While instruction-tuned, it is not designed for conversational interactions.
- Limited safety testing: Phind notes that additional safety testing is required before real-world deployments.