Aleteian/Pathfinder-RP-12B-RU: Merged Language Model
Aleteian/Pathfinder-RP-12B-RU is a 12 billion parameter language model developed by Aleteian, created through a merge of several pre-trained models. This model utilizes the Model Breadcrumbs with TIES merge method, a technique designed to combine the strengths of multiple language models effectively.
Key Merge Details
- Base Model: The merging process used IlyaGusev/saiga_nemo_12b as its foundational base.
- Constituent Models: In addition to the base, the merge incorporated contributions from:
- Merge Method: The specific technique employed was Model Breadcrumbs with TIES, as detailed in the research paper "Model Breadcrumbs with TIES".
- Configuration: The merge was performed with specific weighting and density parameters for each model, aiming to optimize the combined performance. The model uses
bfloat16 dtype and a chatml chat template.
Intended Use
This merged model is suitable for a variety of general language generation tasks, benefiting from the diverse capabilities inherited from its constituent models. Its architecture is designed to provide a robust foundation for applications requiring broad language understanding and generation.