uukuguy/speechless-llama2-hermes-orca-platypus-13b

Model Overview

This model, speechless-llama2-hermes-orca-platypus-13b, is a 13 billion parameter language model created by merging two prominent Llama 2-based models: NousResearch/Nous-Hermes-Llama2-13b and Open-Orca/OpenOrca-Platypus2-13B. It leverages the optimized transformer architecture of Llama 2, which was developed by Meta and trained on 2 trillion tokens of publicly available data with a cutoff of September 2022.

Key Capabilities & Performance

• Merged Architecture: Combines the strengths of Nous-Hermes-Llama2-13b and OpenOrca-Platypus2-13B, both fine-tuned generative text models.
• Benchmark Performance: Achieves an average score of 64.52 across key academic benchmarks, including ARC (60.92), HellaSwag (83.50), MMLU (59.39), and TruthfulQA (54.29).
• Context Length: Supports a context length of 4096 tokens.
• Llama 2 Foundation: Benefits from the extensive pretraining and fine-tuning methodologies of the Llama 2 family, which includes supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) for alignment.

Intended Use Cases

This model is suitable for a variety of natural language generation tasks, particularly those requiring robust performance across general reasoning and knowledge-based queries. It is intended for commercial and research use in English, aligning with the broader Llama 2 ecosystem's guidelines. Developers should perform safety testing and tuning for specific applications.