DreadPoor/Derivative-8B-Model_Stock
DreadPoor/Derivative-8B-Model_Stock is an 8 billion parameter language model with a 32768-token context length, created by DreadPoor using the Model Stock merge method. It is based on FuseAI/FuseChat-Llama-3.1-8B-SFT and integrates three other DreadPoor models. This model is designed for general language tasks, leveraging its merged architecture to combine capabilities from its constituent models.
Loading preview...
Overview
DreadPoor/Derivative-8B-Model_Stock is an 8 billion parameter language model developed by DreadPoor. It was created using the Model Stock merge method, a technique described in the paper Model Stock. The model is built upon FuseAI/FuseChat-Llama-3.1-8B-SFT as its base and incorporates three additional models from DreadPoor: BaeZel_1.1-8B-Model_Stock, Aspire-8B-model_stock, and ONeil-model_stock-8B. This merging approach aims to combine the strengths of multiple pre-trained models into a single, cohesive unit.
Key Capabilities
- Merged Architecture: Leverages the Model Stock method to integrate multiple models, potentially enhancing diverse capabilities.
- Base Model: Built on FuseAI/FuseChat-Llama-3.1-8B-SFT, providing a strong foundation.
- Context Length: Supports a substantial context window of 32768 tokens.
Performance Highlights
Evaluations on the Open LLM Leaderboard show an Average score of 30.04%. Specific metric results include:
- IFEval (0-Shot): 76.67%
- BBH (3-Shot): 34.25%
- MMLU-PRO (5-shot): 31.23%
Good for
- Experimenting with models created via the Model Stock merging technique.
- Applications requiring a model with a broad base of knowledge from its constituent merged models.
- Tasks benefiting from a large context window (32768 tokens).