Hugging Face Overview
TinyllamaMix-1.1B Overview
Aryanne/TinyllamaMix-1.1B is an experimental 1.1 billion parameter language model based on the TinyLlama architecture. It distinguishes itself through a custom merge method, combining both task-swapping and task-arithmetic techniques. This approach integrates components from models like TinyDolphin-2.8.2-1.1b-laser and TinyLlama-1.1B-Chat-v1.0, with a specific focus on enhancing role-playing capabilities.
Key Characteristics
- Architecture: Built on the efficient TinyLlama base, offering a compact 1.1 billion parameters.
- Merge Method: Employs a unique combination of task-swapping and task-arithmetic merges, suggesting a specialized optimization strategy.
- Context Length: Supports a context window of 2048 tokens.
- Performance: Achieves an average score of 32.99 on the Open LLM Leaderboard, with specific scores including 48.39 on HellaSwag (10-Shot) and 58.48 on Winogrande (5-shot).
Potential Use Cases
- Role-Playing (RP): Explicitly designed and optimized for improved performance in role-playing applications.
- Resource-Constrained Environments: Its small parameter count makes it suitable for deployment where computational resources are limited.
- Experimental Merging: Useful for researchers and developers interested in exploring advanced model merging techniques.