maldv/winter-garden-7b-delta: A Conversational Base Model
maldv/winter-garden-7b-delta is an experimental 7 billion parameter language model developed by maldv. It is constructed using an iterative DARE-TIES tree merge method, starting with Mistral-7B-v0.1 and integrating a diverse set of fine-tuned models. This unique merging approach aims to combine the strengths of various specialized models into a cohesive base.
Key Capabilities & Design:
- Iterative Merge Strategy: Utilizes a DARE-TIES tree merge, ordering models by tensor-relative cosine similarity for integration.
- Conversational Focus: Specifically designed to excel in multi-turn conversations, making it a strong foundation for applications requiring sustained dialogue.
- Chat Template Compatibility: Adheres to a '' ended turn chat template, ensuring high compatibility with standard chat interfaces and training methodologies.
Performance Highlights:
The model demonstrates solid performance across several benchmarks:
- Average Score: 64.93
- HellaSwag: 84.37
- MMLU: 60.38
- TruthfulQA: 67.95
Intended Use Case:
This model is primarily intended as a base model for further training and fine-tuning, particularly for tasks involving long-form, multi-turn conversational AI. Its design prioritizes robust dialogue capabilities, making it suitable for developers looking to build specialized conversational agents.