Undi95/Nous-Hermes-13B-Code

TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kPublished:Sep 2, 2023License:cc-by-nc-4.0Architecture:Transformer0.0K Open Weights Cold

Undi95/Nous-Hermes-13B-Code is a 13 billion parameter language model based on a merge of NousResearch/Nous-Hermes-Llama2-13b and a code-focused LoRA from jondurbin/airoboros-lmoe-13b-2.1. This model is specifically designed for code-related tasks, leveraging its merged architecture to enhance programming capabilities. It achieves an average score of 51.98 on the Open LLM Leaderboard, indicating its general performance across various benchmarks.

Loading preview...

Undi95/Nous-Hermes-13B-Code Overview

Undi95/Nous-Hermes-13B-Code is a 13 billion parameter language model developed by Undi95, created by merging two distinct models: NousResearch/Nous-Hermes-Llama2-13b and a LoRA adapter from jondurbin/airoboros-lmoe-13b-2.1. The merge was performed with a 0.30 weight applied to the code-focused LoRA, indicating an intentional emphasis on improving code generation and understanding capabilities.

Key Capabilities & Performance

This model is primarily optimized for tasks requiring strong coding proficiency, benefiting from its specialized merged architecture. Its performance has been evaluated on the Open LLM Leaderboard, where it achieved an average score of 51.98. Specific benchmark results include:

  • ARC (25-shot): 61.18
  • HellaSwag (10-shot): 83.21
  • MMLU (5-shot): 55.13
  • TruthfulQA (0-shot): 50.56
  • Winogrande (5-shot): 75.14
  • GSM8K (5-shot): 10.39
  • DROP (3-shot): 28.28

These scores provide insight into its general reasoning, common sense, and language understanding, alongside its mathematical and reading comprehension abilities. The model's 4096-token context length supports processing moderately long inputs for various applications.

Good For

  • Code generation and completion: The model's architecture, incorporating a code-specific LoRA, suggests suitability for programming tasks.
  • General language understanding: Its foundation in Nous-Hermes-Llama2-13b provides a solid base for diverse NLP applications.
  • Experimentation with merged models: Developers interested in exploring the performance characteristics of merged language models, particularly those with a focus on specialized domains like code.

Finetunes
2 models