Hugging Face KansenSakura-Symbiosis-12B Overview
KansenSakura-Symbiosis-12B is an experimental 12 billion parameter language model developed by Valeciela, featuring a substantial 32768 token context window. This model is a product of a Multi-SLERP merge process, building upon Retreatcost/KansenSakura-Zero-RP-12b as its foundational base.
Merge Details
The model integrates contributions from three distinct KansenSakura-RP-12b variants:
Retreatcost/KansenSakura-Erosion-RP-12bRetreatcost/KansenSakura-Eclipse-RP-12bRetreatcost/KansenSakura-Radiance-RP-12b
Each of these models contributed with a weight of 0.5 during the Multi-SLERP merge, aiming to combine their respective strengths. The configuration specifies a chatml chat template and bfloat16 dtype, with normalization applied during the merge.
Usage and Availability
This model is provided in GGUF format, with both a standard version and a Q6_K_XL quantized version available for download. Users are advised that this is an experimental model and should be used with caution.