Novaciano/Uncensored_Varuna-3.2-1B
Novaciano/Uncensored_Varuna-3.2-1B is a 1 billion parameter language model created by Novaciano, merged using the Arcee Fusion method with cazzz307/Abliterated-Llama-3.2-1B-Instruct as its base. This model integrates components from Novaciano/Eurinoferus-3.2-1B, focusing on specific weight adjustments for MLP and lm_head layers. It offers a 32768 token context length, making it suitable for applications requiring moderate context understanding and efficient inference.
Loading preview...
Model Overview
Novaciano/Uncensored_Varuna-3.2-1B is a 1 billion parameter language model developed by Novaciano. It was created through a merge operation using mergekit, specifically employing the Arcee Fusion method.
Merge Details
The model's foundation is cazzz307/Abliterated-Llama-3.2-1B-Instruct. It incorporates elements from Novaciano/Eurinoferus-3.2-1B through a strategic merge configuration. This configuration involved specific weight adjustments, applying different values to the MLP (Multi-Layer Perceptron) and lm_head layers during the fusion process. The merge was performed with float32 dtype and an out_dtype of bfloat16.
Key Characteristics
- Parameter Count: 1 billion parameters.
- Context Length: Supports a context window of 32768 tokens.
- Merge Method: Utilizes the Arcee Fusion technique for combining pre-trained models.
Potential Use Cases
This model is designed for applications that benefit from a compact yet capable language model with a decent context window. Its merged architecture suggests a focus on combining the strengths of its constituent models, potentially offering improved performance for instruction-following or specific domain tasks where the base and merged models excel.