Undi95/Toppy-M-7B: A Merged 7B Language Model

Toppy-M-7B is a 7 billion parameter model developed by Undi95, created using the task_arithmetic merge method from MergeKit. This approach combines the weights of multiple pre-existing models and LoRAs to synthesize a new model with enhanced capabilities.

Key Components and Merge Strategy

This model is a sophisticated blend of several well-regarded Mistral-based models and their associated LoRAs, specifically:

• openchat/openchat_3.5 combined with lemonilia/AshhLimaRP-Mistral-7B (LoRA)
• NousResearch/Nous-Capybara-7B-V1.9 combined with Vulkane/120-Days-of-Sodom-LoRA-Mistral-7b
• HuggingFaceH4/zephyr-7b-beta combined with Undi95/Mistral-pippa-sharegpt-7b-qlora

The task_arithmetic merge method allows for fine-grained control over the contribution of each component, aiming to integrate their respective strengths. The base model for this merge is mistralai/Mistral-7B-v0.1.

Intended Use Cases

Given its composite nature, Toppy-M-7B is designed to be a versatile model, potentially excelling in areas where its constituent models perform well. It utilizes an Alpaca prompt template, making it suitable for instruction-following tasks and general conversational AI applications. Its 7B parameter count and 4096-token context window make it a strong candidate for deployment in environments requiring efficient yet capable language generation.