RWKV/v6-Finch-7B-World3-HF
The RWKV/v6-Finch-7B-World3-HF is a 7 billion parameter model from the RWKV architecture family, developed by RWKV. This Hugging Face compatible model demonstrates improved performance over its predecessor, Eagle 7B, across various benchmarks including MMLU and HellaSwag. It is designed for general language understanding and generation tasks, offering enhanced reasoning capabilities.
Loading preview...
RWKV v6 Finch 7B World3 Model
This is the Hugging Face Transformers implementation of the Finch 7B World3 model, part of the RWKV architecture family. It represents an advancement over the previous Eagle 7B model, offering improved performance across several key benchmarks.
Key Capabilities & Performance
The Finch 7B World3 model shows notable gains in reasoning and general knowledge tasks compared to Eagle 7B. Evaluation results highlight its enhanced capabilities:
- ARC: Improved from 39.59% (Eagle 7B) to 41.47% (Finch 7B).
- HellaSwag: Increased from 53.09% (Eagle 7B) to 55.96% (Finch 7B).
- MMLU: Significantly improved from 30.86% (Eagle 7B) to 41.70% (Finch 7B).
- Truthful QA: Rose from 33.03% (Eagle 7B) to 34.82% (Finch 7B).
- Winogrande: Enhanced from 67.56% (Eagle 7B) to 71.19% (Finch 7B).
These improvements indicate a more robust model for understanding and generating human-like text, with better performance on common sense reasoning and factual recall.
Usage
The model is designed for seamless integration with the Hugging Face Transformers library, supporting both CPU and GPU inference. Example code snippets are provided for quick setup and generation, including batch inference capabilities. The development of this model was supported by Recursal.ai for GPU resources and training management, and EleutherAI for their contributions to the v5/v6 Eagle/Finch paper.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.