Aleteian/Pathfinder-RP-12B-RU
Aleteian/Pathfinder-RP-12B-RU is a 12 billion parameter language model created by Aleteian, merged using the Model Breadcrumbs with TIES method. It is based on IlyaGusev/saiga_nemo_12b and incorporates MarinaraSpaghetti/NemoMix-Unleashed-12B and LatitudeGames/Wayfarer-12B. This model is designed for general language generation tasks, leveraging the combined strengths of its constituent models.
Loading preview...
Aleteian/Pathfinder-RP-12B-RU: Merged Language Model
Aleteian/Pathfinder-RP-12B-RU is a 12 billion parameter language model developed by Aleteian, created through a merge of several pre-trained models. This model utilizes the Model Breadcrumbs with TIES merge method, a technique designed to combine the strengths of multiple language models effectively.
Key Merge Details
- Base Model: The merging process used IlyaGusev/saiga_nemo_12b as its foundational base.
- Constituent Models: In addition to the base, the merge incorporated contributions from:
- Merge Method: The specific technique employed was Model Breadcrumbs with TIES, as detailed in the research paper "Model Breadcrumbs with TIES".
- Configuration: The merge was performed with specific weighting and density parameters for each model, aiming to optimize the combined performance. The model uses
bfloat16dtype and achatmlchat template.
Intended Use
This merged model is suitable for a variety of general language generation tasks, benefiting from the diverse capabilities inherited from its constituent models. Its architecture is designed to provide a robust foundation for applications requiring broad language understanding and generation.