OrobasVault/BROKEN_MERGE_TensorGuard-Prototype-24B-v1
OrobasVault/BROKEN_MERGE_TensorGuard-Prototype-24B-v1 is a 24 billion parameter language model based on the MistralForCausalLM architecture, created using the experimental TensorGuard merge method. This prototype merge combines several 24B models, including those from Naphula and TheDrummer, with a 32768 token context length. It is explicitly noted as producing broken output and is not recommended for use, serving primarily as a demonstration of the TensorGuard merging technique. The model's configuration details the use of various noise strategies and similarity metrics for the merge process.
Loading preview...
TensorGuard-Prototype-24B-v1 Overview
This model, OrobasVault/BROKEN_MERGE_TensorGuard-Prototype-24B-v1, is a 24 billion parameter language model built on the MistralForCausalLM architecture. It was created using mergekit and specifically employs the experimental TensorGuard merge method, as detailed in arXiv:2506.01631v2.
Key Characteristics
- Experimental Merge Method: Utilizes the TensorGuard technique, which involves specific parameters like
noise_epsilon,num_perturbations, and variousnoise_strategies(adversarial, structural, low_freq, high_freq, gaussian). - Merged Components: Combines several 24B base models, including
/workspace/Naphula--BeaverAI_Fallen-Mistral-Small-3.1-24B-v1e_textonly,/workspace/TheDrummer--Magidonia-24B-v4.3,/workspace/TheDrummer--Precog-24B-v1, and/workspace/TheDrummer--Cydonia-24B-v4.3. - Context Length: Supports a context window of 32768 tokens.
- Explicitly Broken: A critical note from the creator states that this merge produces BROKEN output and is not recommended for download, indicating the TensorGuard method requires revision.
Intended Use
This model is primarily intended for:
- Research and Development: Studying the effects and limitations of the TensorGuard merge method.
- Debugging Merge Techniques: Analyzing why this specific TensorGuard configuration leads to broken outputs.
It is not recommended for general use due to its known broken state.