zarakiquemparte/zarafusionix-l2-7b
The zarakiquemparte/zarafusionix-l2-7b is a 7 billion parameter language model created by zarakiquemparte, formed by merging Nous Hermes Llama2 7b (62%) and Stable Beluga 7b (38%), then further merged with LimaRP LLama2 7B Lora. This fusion aims to combine the strengths of its constituent models, offering enhanced performance across various instruction formats. It is particularly notable for its unique merging methodology, which allows for reproduction using provided scripts.
Loading preview...
Zarafusionix L2 7b: Merged Language Model
Zarafusionix L2 7b is a 7 billion parameter language model developed by zarakiquemparte, distinguished by its unique merging strategy. It combines three prominent models:
- Nous Hermes Llama2 7b (62% contribution)
- Stable Beluga 7b (38% contribution)
- LimaRP LLama2 7B Lora (applied as a LoRA merge)
This fusion process was executed using custom scripts, which are publicly available, allowing for full reproducibility of the model's creation. The model leverages the combined capabilities of its base components, aiming for a versatile performance profile.
Key Capabilities & Instruction Formats
Due to its merged heritage, Zarafusionix L2 7b supports multiple instruction formats, making it adaptable to various prompting styles:
- Alpaca 2 Format:
### Instruction: <prompt> ### Response: <leave a newline blank for model to respond> - LimaRP Instruction Format:
<<SYSTEM>> <character card and system prompt> <<USER>> <prompt> <<AIBOT>> <leave a newline blank for model to respond>
Limitations
It is important to note that this model is not designed for providing factual information or advice. Users should exercise caution and verify any information generated.
Training Details
The model's creation involved a merging process rather than traditional training from scratch. The specific tools and scripts used for merging are provided, enabling users to understand and replicate the model's construction.