NewstaR/7B-Orfini
NewstaR/7B-Orfini is an experimental 7 billion parameter language model created by NewstaR through a custom merge of StableBeluga-7B, orca_mini_v3_7b, and Marcoroni-7B. This model is intended for testing and research purposes to evaluate the outcomes of merging diverse foundation models without further fine-tuning. Its primary characteristic is its experimental nature, designed to explore the stability and performance implications of architectural and weight integration.
Loading preview...
Orfini: An Experimental Merged Model
Orfini is an experimental 7 billion parameter model developed by NewstaR, created by merging the weights and architectures of three distinct foundation models: stabilityai/StableBeluga-7B, pankajmathur/orca_mini_v3_7b, and AIDC-ai-business/Marcoroni-7B. This model was constructed using a custom merging technique, with no additional fine-tuning performed post-merge.
Key Characteristics & Purpose
- Experimental Nature: Primarily intended for testing and research to understand the effects of model merging.
- Foundation Models: Inherits characteristics and potential biases from its constituent models, which were trained on datasets like COT, Niv2, t0, FLAN2021, Dolphin, and OpenOrca.
- Untested Capabilities: As a newly merged model, its specific capabilities and limitations are currently unknown and require extensive evaluation.
Limitations and Considerations
Due to its experimental status, Orfini presents several limitations:
- Potential Instability: Merging diverse architectures may lead to unpredictable behavior.
- Compounded Biases: It may exhibit biases and issues present in all three original foundation models.
- Performance Variability: Performance on various tasks might be decreased compared to its individual foundation models.
Orfini is not recommended for production systems or public-facing content generation. Ethical use is paramount, given its untested nature and potential for unreliable or biased outputs.