Kukedlc/Neural-4-ARC-7b
Kukedlc/Neural-4-ARC-7b is a 7 billion parameter language model created by Kukedlc, formed by merging five distinct models using the dare_ties method. This model integrates components from paulml/OmniBeagleSquaredMBX-v3-7B, nlpguy/AlloyIngotNeoX, Gille/StrangeMerges_21-7B-slerp, Kukedlc/Jupiter-k-7B-slerp, and Kukedlc/NeuralSirKrishna-7b. With a context length of 8192 tokens, it is designed to leverage the combined strengths of its constituent models for general language generation tasks.
Loading preview...
Model Overview
Kukedlc/Neural-4-ARC-7b is a 7 billion parameter language model developed by Kukedlc. It is a product of a sophisticated merge operation, combining five different base models using the dare_ties merge method via LazyMergekit. This approach aims to synthesize the capabilities of multiple specialized models into a single, more versatile entity.
Key Components and Merge Strategy
The model integrates contributions from:
- Kukedlc/NeuralSirKrishna-7b (serving as the base model and a merged component)
- paulml/OmniBeagleSquaredMBX-v3-7B
- nlpguy/AlloyIngotNeoX
- Gille/StrangeMerges_21-7B-slerp
- Kukedlc/Jupiter-k-7B-slerp
The dare_ties merge method, along with specific density and weight parameters for each contributing model, was used to create Neural-4-ARC-7b. This configuration suggests an effort to balance and combine the strengths of its diverse origins.
Usage and Technical Details
Neural-4-ARC-7b supports a context length of 8192 tokens and is configured to use bfloat16 for its dtype. It includes int8_mask in its parameters. The model can be easily integrated into Python projects using the transformers library, with example code provided for text generation tasks, demonstrating how to load the model and tokenizer, apply chat templates, and generate outputs.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.