Eric111/MarcoHermes
MarcoHermes is a 7 billion parameter language model created by Eric111, formed by merging AtAndDev/CapybaraMarcoroni-7B and eren23/DistilHermes-2.5-Mistral-7B using the mergekit tool. This model leverages the strengths of its constituent models, offering a 4096-token context length. It is designed to combine the capabilities of its base models, making it suitable for general-purpose language generation tasks.
Loading preview...
Model Overview
MarcoHermes is a 7 billion parameter language model developed by Eric111. It is a merged model, created using the mergekit tool, combining two distinct base models: AtAndDev/CapybaraMarcoroni-7B and eren23/DistilHermes-2.5-Mistral-7B.
Key Characteristics
- Architecture: A blend of two 7B parameter models, leveraging their respective strengths.
- Merge Method: Utilizes the
slerp(spherical linear interpolation) merge method, with specific parameter weighting applied to self-attention and MLP layers to optimize performance. - Context Length: Supports a context window of 4096 tokens.
- Data Type: Configured to use
bfloat16for efficient computation.
Use Cases
This model is suitable for general language generation tasks, benefiting from the combined capabilities of its merged components. Developers looking for a 7B model that integrates features from both CapybaraMarcoroni and DistilHermes-2.5-Mistral may find MarcoHermes a compelling option for various NLP applications.