Orfini: An Experimental Merged Model
Orfini is an experimental 7 billion parameter model developed by NewstaR, created by merging the weights and architectures of three distinct foundation models: stabilityai/StableBeluga-7B, pankajmathur/orca_mini_v3_7b, and AIDC-ai-business/Marcoroni-7B. This model was constructed using a custom merging technique, with no additional fine-tuning performed post-merge.
Key Characteristics & Purpose
- Experimental Nature: Primarily intended for testing and research to understand the effects of model merging.
- Foundation Models: Inherits characteristics and potential biases from its constituent models, which were trained on datasets like COT, Niv2, t0, FLAN2021, Dolphin, and OpenOrca.
- Untested Capabilities: As a newly merged model, its specific capabilities and limitations are currently unknown and require extensive evaluation.
Limitations and Considerations
Due to its experimental status, Orfini presents several limitations:
- Potential Instability: Merging diverse architectures may lead to unpredictable behavior.
- Compounded Biases: It may exhibit biases and issues present in all three original foundation models.
- Performance Variability: Performance on various tasks might be decreased compared to its individual foundation models.
Orfini is not recommended for production systems or public-facing content generation. Ethical use is paramount, given its untested nature and potential for unreliable or biased outputs.