alchemonaut/QuartetAnemoi-70B-t0.0001
QuartetAnemoi-70B-t0.0001 is a 69 billion parameter language model created by alchemonaut, formed by sequentially merging four 70B models (miqu-1-70b-sf, WinterGoddess-1.4x-70B-L2, Aurora-Nights-70B-v1.0, Xwin-LM-70B-V0.1) using a custom NearSwap algorithm with a low 't' parameter of 0.0001. This model is noted for its storytelling capabilities, particularly its avoidance of common narrative clichés. It achieves an average score of 76.86 on the Open LLM Leaderboard, with a 75.42 on MMLU and 68.61 on GSM8k.
Loading preview...
QuartetAnemoi-70B-t0.0001 Overview
QuartetAnemoi-70B-t0.0001 is a 69 billion parameter language model developed by alchemonaut. It is the result of a sequential merge of four distinct 70B models: miqu-1-70b-sf, WinterGoddess-1.4x-70B-L2, Aurora-Nights-70B-v1.0, and Xwin-LM-70B-V0.1. This merge was performed using a proprietary NearSwap algorithm with a very low 't' parameter of 0.0001, which ensures that only about 0.8% of weights are fully switched during each pass, leading to "extremely soft" changes.
Key Capabilities & Characteristics
- Storytelling Prowess: The model demonstrates strong storytelling abilities, notably avoiding common narrative clichés like "In the end" or "And so."
- Unique Merging Technique: Utilizes the NearSwap algorithm, which interpolates weights between models when they are similar, retaining most of the base model's characteristics while subtly integrating others.
- Performance: Achieves an average score of 76.86 on the Open LLM Leaderboard, including 75.42 on MMLU and 68.61 on GSM8k.
Use Cases & Considerations
- Creative Writing: Particularly well-suited for generating narrative content and stories due to its unique output style.
- Research: Licensed for noncommercial research use only, given the unknown origin of its base model, Miqu.
- Quantization: Available in various quantization formats, including GGUF and EXL2, for broader accessibility and deployment.