ghost4280/Ghost-V4-Custom-8B
Ghost-V4-Custom-8B is an 8 billion parameter language model created by ghost4280, leveraging the TIES merge method. It is based on unsloth/DeepSeek-R1-Distill-Llama-8B and incorporates NousResearch/Hermes-2-Pro-Mistral-7B, offering a 32768-token context length. This model is designed for general language tasks, combining the strengths of its merged components.
Loading preview...
Model Overview
Ghost-V4-Custom-8B is an 8 billion parameter language model developed by ghost4280. This model was created using the TIES (Trimmed, Iterative, and Selective) merge method, a technique designed to combine the strengths of multiple pre-trained language models efficiently. It utilizes unsloth/DeepSeek-R1-Distill-Llama-8B as its foundational base model.
Key Components and Merge Details
The model integrates NousResearch/Hermes-2-Pro-Mistral-7B alongside its base, aiming to leverage the distinct capabilities of each. The merge process assigned a weight of 1.0 to the DeepSeek-R1-Distill-Llama-8B base and a weight of 0.5 to Hermes-2-Pro-Mistral-7B, indicating a primary reliance on the DeepSeek architecture while incorporating specific features from Hermes-2-Pro. The configuration specified a density of 0.5 and used bfloat16 for its data type, with the tokenizer sourced from the base model.
Potential Use Cases
Given its merged architecture, Ghost-V4-Custom-8B is suitable for a variety of general-purpose language generation and understanding tasks. Its 32768-token context length allows for processing longer inputs and generating more coherent, extended outputs. Developers looking for a model that combines the characteristics of DeepSeek and Hermes-2-Pro in an 8B parameter footprint may find this model particularly useful.