alnrg2arg/blockchainlabs_7B_merged_test2_4_prune
alnrg2arg/blockchainlabs_7B_merged_test2_4_prune is a 7 billion parameter pruned language model based on the Mistral architecture, derived from a merge of mlabonne/NeuralBeagle14-7B and udkai/Turdus. This model utilizes the wanda pruning technique to optimize its structure while maintaining capabilities from its merged base models. It is designed for general language tasks, leveraging its 8192-token context length for efficient processing.
Loading preview...
Model Overview
alnrg2arg/blockchainlabs_7B_merged_test2_4_prune is a 7 billion parameter language model built upon the Mistral architecture. It is a pruned version of alnrg2arg/blockchainlabs_7B_merged_test2_4, which itself was created by merging two distinct models: mlabonne/NeuralBeagle14-7B and udkai/Turdus using MergeKit.
Key Characteristics
- Pruned Architecture: The model has undergone pruning using the Wanda technique, aiming for efficiency while retaining performance.
- Base Models: Inherits characteristics from its merged predecessors, NeuralBeagle14-7B and Turdus.
- Context Length: Features a maximum position embedding of 32768, with a sliding window of 4096, indicating a practical context window of 8192 tokens.
- Mistral-based: Leverages the efficient and performant Mistral architecture.
Use Cases
This model is suitable for a variety of general-purpose natural language processing tasks where a 7B parameter model with an optimized, pruned structure is beneficial. Its merged origins suggest a broad range of potential applications, from text generation and summarization to question answering, within its 8192-token context limit.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.