Overview

The h2m/mhm-7b-v1.3 is a 7 billion parameter experimental language model developed by h2m. It was created using mergekit by combining seven distinct pre-trained language models, primarily leveraging the Mistral architecture. The merging process involved three stages, integrating models that were highly ranked on the OpenLLM leaderboard.

Key Characteristics

• Architecture: Based on the Mistral family of models.
• Parameter Count: 7 billion parameters.
• Context Length: Supports a context window of 4096 tokens.
• Development Method: Utilizes mergekit for combining multiple models.
• Origin: A merge of seven models, selected from the top of the OpenLLM leaderboard.

Intended Use

This model is primarily an experimental merge designed to explore the capabilities and performance characteristics of combining multiple high-performing language models. It serves as a testbed for understanding how different model components interact when merged. Users interested in model merging techniques or exploring the emergent properties of combined LLMs may find this model particularly relevant for research and experimentation.