ChaoticNeutrals/Domain-Fusion-L3-8B

Model Overview

Domain-Fusion-L3-8B is an 8 billion parameter language model developed by ChaoticNeutrals, created through a strategic merging process. This model is a fusion of two internal lineage models: Hathor_Variant-X and T-900xBioLLM. The merging was performed using a slerp (spherical linear interpolation) method, specifically targeting different weights for self-attention (self_attn) and multi-layer perceptron (mlp) components across layers.

Key Merging Details

• Source Models: Hathor_Variant-X and T-900xBioLLM.
• Merge Method: Slerp, applied across layers 0 to 32 of both source models.
• Parameter Weights: Distinct interpolation values were used for self_attn (ranging from 0 to 1) and mlp (ranging from 1 to 0) filters, with a general t value of 0.5 for other parameters.
• Data Type: The model was produced using bfloat16 precision.

Recommended Use

This model is suitable for general language generation and understanding tasks, inheriting capabilities from its merged predecessors. Users can find recommended SillyTavern (ST) presets for optimal performance with this model at Domain Fusion Presets.