Undi95/PsyMedRP-v1-13B
Undi95/PsyMedRP-v1-13B is a 13 billion parameter language model created by Undi95, developed through a series of merges combining various base models including airoboros-l2-13b, Samantha-1.11-13b, Xwin-LM-13B-V0.1, MedLLaMA_13B, and Huginn-13b. This model is a complex merge designed for specific, yet currently unspecified, applications, with further development and testing underway. Its unique multi-stage merging process suggests an intent to combine diverse capabilities from its constituent models.
Loading preview...
Model Overview
Undi95/PsyMedRP-v1-13B is a 13 billion parameter language model developed by Undi95, distinguished by its intricate multi-stage merging architecture. This model is the result of a progressive fusion of several established base models, each contributing distinct characteristics to the final iteration.
Merging Architecture
The development of PsyMedRP-v1-13B involved a four-phase merging process:
- Phase 1 (PsyMedRP-v1-13B-p1): A merge of
jondurbin/airoboros-l2-13b-3.0(85% weight) andehartford/Samantha-1.11-13b(15% weight). - Phase 2 (PsyMedRP-v1-13B-p2): A merge of
Xwin-LM/Xwin-LM-13B-V0.1(85% weight) andchaoyi-wu/MedLLaMA_13B(15% weight). - Phase 3 (PsyMedRP-v1-13B-p3): A subsequent merge combining
PsyMedRP-v1-13B-p1(55% weight) andPsyMedRP-v1-13B-p2(45% weight). - Phase 4 (PsyMedRP-v1-13B-p4): This phase involved merging
The-Face-Of-Goonery/Huginn-13b-FP16with a Gryphe gradient and thePsyMedRP-v1-13B-p3model.
Finally, the Undi95/LimaRP-v3-120-Days model was applied at a 0.3 weight to PsyMedRP-v1-13B-p4 to create the final PsyMedRP-v1-13B model. This complex merging strategy suggests an effort to integrate diverse linguistic and potentially specialized capabilities from its constituent models. The model is currently undergoing testing, with a 20B parameter version planned for future release.