CobraMamba/mamba-gpt-7b-v1: An Instruction-Tuned OpenLLaMA Model

CobraMamba/mamba-gpt-7b-v1 is a 7 billion parameter language model built upon the openlm-research/open_llama_7b_v2 base model. This model has undergone extensive instruction fine-tuning across a variety of datasets, including Stanford Alpaca, Open Assistant (multilingual), LIMA, CodeAlpaca 20k, GPT-4 Generated Data, and UltraChat.

Key Capabilities & Performance

• Enhanced Performance: Through fine-tuning, mamba-gpt-7b-v1 surpasses the original OpenLLaMA model in various evaluation subtasks, achieving performance comparable to other 7B models.
• Instruction Following: Optimized for understanding and executing instructions from diverse sources.
• Multilingual Data Exposure: Training on datasets like Open Assistant (multilingual) contributes to its versatility.
• Benchmark Results: On the Open LLM Leaderboard, it achieves an average score of 54.77, with notable scores including 61.26 on ARC (25-shot) and 84.1 on HellaSwag (10-shot).

Good For

• General-purpose conversational AI applications.
• Instruction-based text generation and completion.
• Tasks requiring understanding and response generation based on diverse prompts.
• Developers seeking a capable 7B model with strong instruction-following abilities.