lemon07r/llama-3-NeuralMahou-8b

TEXT GENERATIONConcurrency Cost:1Model Size:8BQuant:FP8Ctx Length:8kPublished:May 30, 2024License:llama3Architecture:Transformer0.0K Cold

lemon07r/llama-3-NeuralMahou-8b is an 8 billion parameter language model based on the Llama 3 architecture, created by merging several pre-trained models using the Model Stock method. This model integrates components from nbeerbower/llama-3-spicy-abliterated-stella-8B and flammenai/Mahou-1.2-llama3-8B, using mlabonne/NeuralDaredevil-8B-abliterated as a base. It demonstrates strong general reasoning capabilities, achieving an average score of 71.33 on the Open LLM Leaderboard, making it suitable for a variety of general-purpose language tasks.

Loading preview...

lemon07r/llama-3-NeuralMahou-8b Overview

This model is an 8 billion parameter language model built upon the Llama 3 architecture. It was developed by lemon07r through a sophisticated merge of multiple pre-trained models using the Model Stock method, facilitated by mergekit.

Key Characteristics

  • Architecture: Llama 3-based, 8 billion parameters.
  • Merge Method: Utilizes the Model Stock technique, which combines layers from different source models.
  • Base Model: mlabonne/NeuralDaredevil-8B-abliterated served as the foundational model for the merge.
  • Component Models: Integrates nbeerbower/llama-3-spicy-abliterated-stella-8B and flammenai/Mahou-1.2-llama3-8B to enhance its capabilities.
  • Context Length: Supports an 8192-token context window.

Performance Highlights

Evaluated on the Open LLM Leaderboard, llama-3-NeuralMahou-8b achieved an average score of 71.33. Notable scores include:

  • AI2 Reasoning Challenge (25-Shot): 67.41
  • HellaSwag (10-Shot): 83.45
  • MMLU (5-Shot): 68.63
  • GSM8k (5-Shot): 72.55

Ideal Use Cases

This model is well-suited for general-purpose applications requiring strong reasoning and language understanding, given its balanced performance across various benchmarks. Its merged nature suggests a blend of capabilities from its constituent models, making it adaptable for tasks like:

  • General text generation and completion.
  • Question answering and summarization.
  • Reasoning-intensive tasks, as indicated by its AI2 Reasoning Challenge and GSM8k scores.

Finetunes
2 models