OrobasVault/Geodesic-Phantom-12B
OrobasVault/Geodesic-Phantom-12B is a 12 billion parameter language model created by OrobasVault, merged using the karcher_stock method from a Mistral-Nemo-Instruct-2407 base and six Vortex5 12B models. This model leverages an adaptive VRAM chunking script for efficient merging and incorporates a karcher_stock adaptive tanh soft-clamp to prevent merge corruption. It is designed for general language generation tasks, benefiting from the combined strengths of its constituent models.
Loading preview...
Geodesic Phantom 12B Overview
Geodesic Phantom 12B is a 12 billion parameter language model developed by OrobasVault. It was created through a sophisticated merge process using MergeKit, specifically employing the karcher_stock method. The base model for this merge was mistralai--Mistral-Nemo-Instruct-2407, combined with six distinct 12B models from Vortex5, including Wicked-Nebula-12B, Celestial-Queen-12B, Moonlit-Mirage-12B, Stellar-Witch-12B, Prototype-X-12b, and Crimson-Constellation-12B.
Key Technical Details
- Merge Method: Utilizes the
karcher_stockmerge method, known for its robust approach to combining model weights. - Adaptive Soft-Clamp: A notable feature is the implementation of a
karcher_stockAdaptive Tanh Soft-Clamp (v11) during the merge process. This patch is crucial for preventing merge corruption, particularly when dealing with potential negative infinity spikes in the t-factor calculation, ensuring a more stable and reliable merged model. - Efficient Merging: The model was merged efficiently on a Runpod A40, leveraging an adaptive VRAM chunking script to manage memory during the intensive merging operation.
Potential Use Cases
Given its foundation in instruction-tuned models and a diverse set of merged components, Geodesic Phantom 12B is likely suitable for a broad range of general-purpose language generation tasks, including but not limited to:
- Text completion and generation
- Instruction following
- Creative writing assistance
- Conversational AI applications