WhoTookMyAmogusNickname/NewHope_HF_not_official
NewHope is a 13 billion parameter instruction-tuned chat model developed by SLAM (SUFE Large AI Model) group, based on the Llama-2 architecture. It is specifically fine-tuned for strong coding capabilities across multiple languages including Python, C++, Java, JavaScript, and Go. The model achieves a Pass@1 score of 66.5 on HumanEval, demonstrating high performance in code generation tasks. This model is particularly suited for developers requiring robust code completion and generation.
Loading preview...
NewHope: A Code-Focused Llama-2 Fine-tune
NewHope is a 13 billion parameter instruction-tuned chat model, developed by the SLAM (SUFE Large AI Model) research group at Shanghai University of Finance and Economics. It is built upon the Llama-2 architecture and is specifically designed to excel in programming tasks.
Key Capabilities & Performance
- Multi-language Code Generation: Supports various programming languages including Python, C++, Java, JavaScript, and Go.
- High HumanEval Performance: Achieves a Pass@1 score of 66.5 on the HumanEval benchmark, closely approaching GPT-4's score of 67.0 in coding ability.
- Instruction Following: Capable of generating code based on natural language instructions and engaging in dialog-based interactions for code-related queries.
Noteworthy Information
- Evaluation Data Leak: The developers explicitly state that evaluation data has leaked into the dataset used for this model.
- Open-sourced Weights: Model weights are available on Hugging Face under the SLAM-group/NewHope repository.
- Leaderboard Performance: On the Open LLM Leaderboard, NewHope shows an average score of 51.9, with notable scores in HellaSwag (84.03) and Winogrande (74.98), while GSM8K (15.85) and DROP (26.66) are lower.
Usage
NewHope can be loaded using the Hugging Face Transformers library (version 4.31.0 or higher) and supports both single-turn instruction-based code generation and multi-turn conversational interactions.