Model Overview
Lamarck-14B-v0.7-Fusion is an experimental 14.8 billion parameter language model developed by sometimesanotion, featuring a substantial 131,072 token context length. This model is notable for its complex, multi-stage fusion merge strategy, which combines several pre-existing models and merges, including Lamarck-14B-v0.7, Lamarckvergence-14B, Qwenvergence-14B-v12-Prose-DS, and Chocolatine-2-14B-Instruct-v2.0.3. The primary goal of this merge was to investigate the behavior of multiple fusion merges.
Key Capabilities
- Strong Prose Generation: The model demonstrates a particular strength in generating high-quality, free-form prose, attributed to its re-emphasis of Qwenvergence-14B-v12-Prose-DS during the merging process.
- High GPQA and Reasoning: Evaluations indicate high performance on GPQA benchmarks and robust reasoning capabilities.
- Experimental Merge Strategy: It serves as a testbed for understanding how multiple fusion merges interact and contribute to overall model performance.
Good For
- Free-form Creativity: Ideal for applications requiring creative text generation, storytelling, or other tasks benefiting from nuanced and expressive language.
- Research into Merge Methods: Useful for researchers and developers interested in exploring advanced model merging techniques, particularly multi-stage fusion and SLERP methods.
- Complex Reasoning Tasks: Suitable for tasks that demand strong logical inference and problem-solving, as suggested by its high GPQA scores.