Model Overview
Ba2han/HermesStar-OrcaWind-Synth-11B is a 10.7 billion parameter model developed by Ba2han through a complex series of linear merges. The merging process involved combining several established models, including Open Hermes, Starling, SlimOrca, Zephyr Beta, and Synthia, with a final merge ratio of 1 to 0.3. This unique synthesis aims to integrate the strengths of its constituent models.
Key Characteristics
- Merged Architecture: Combines multiple base models (Open Hermes, Starling, SlimOrca, Zephyr Beta, Synthia) to create a novel blend of capabilities.
- ChatML Compatibility: Primarily designed to be used with the ChatML template, ensuring consistent instruction-following behavior.
- Repetition Penalty: Users may find that increasing the repetition penalty can enhance output quality, though it might introduce stability issues.
- CoT Prompting: The model responds well to Chain-of-Thought (CoT) prompting, even when integrated into the 'system' role within ChatML, which can improve the quality of generated responses.
Usage Considerations
This model is suitable for general conversational AI and instruction-following tasks. Developers are encouraged to experiment with different prompting strategies, particularly CoT, to optimize performance. While the creator notes potential stability issues with higher repetition penalties, careful tuning can yield smarter outputs. The model occasionally produces an extra End-of-Sequence (EOS) token, which users should be aware of.