FrankenVillain-7B-v1 by luqmanxyz is a 10.7 billion parameter language model created through a Franken merge of two instances of jeonsworld/CarbonVillain-en-10.7B-v1 using mergekit. This model is designed for general language tasks, demonstrating an average performance of 45.34 on the Open LLM Leaderboard across various benchmarks. It is suitable for applications requiring a moderately sized model with broad reasoning and language understanding capabilities.
Loading preview...
Overview
FrankenVillain-7B-v1 is a 10.7 billion parameter language model developed by luqmanxyz. It is constructed using a "Franken merge" technique via mergekit, combining two instances of the jeonsworld/CarbonVillain-en-10.7B-v1 model. This merging approach leverages specific layer ranges from the base model to create a new, distinct model.
Key Capabilities & Performance
This model demonstrates general language understanding and reasoning abilities, as evaluated on the Open LLM Leaderboard. Its performance metrics include:
- Avg. Score: 45.34
- AI2 Reasoning Challenge (25-Shot): 42.75
- HellaSwag (10-Shot): 51.52
- MMLU (5-Shot): 48.60
- TruthfulQA (0-shot): 56.19
- Winogrande (5-shot): 73.01
Notably, the model scored 0.00 on the GSM8k (5-shot) benchmark, indicating limitations in complex mathematical reasoning. Detailed evaluation results are available on the Open LLM Leaderboard.
Good For
- General text generation and understanding tasks.
- Applications requiring a 10.7B parameter model with a balanced performance across several common benchmarks.
- Exploration of models created via advanced merging techniques like Franken merge.