Model Overview

Undi95/PsyMedRP-v1-13B is a 13 billion parameter language model developed by Undi95, distinguished by its intricate multi-stage merging architecture. This model is the result of a progressive fusion of several established base models, each contributing distinct characteristics to the final iteration.

Merging Architecture

The development of PsyMedRP-v1-13B involved a four-phase merging process:

• Phase 1 (PsyMedRP-v1-13B-p1): A merge of jondurbin/airoboros-l2-13b-3.0 (85% weight) and ehartford/Samantha-1.11-13b (15% weight).
• Phase 2 (PsyMedRP-v1-13B-p2): A merge of Xwin-LM/Xwin-LM-13B-V0.1 (85% weight) and chaoyi-wu/MedLLaMA_13B (15% weight).
• Phase 3 (PsyMedRP-v1-13B-p3): A subsequent merge combining PsyMedRP-v1-13B-p1 (55% weight) and PsyMedRP-v1-13B-p2 (45% weight).
• Phase 4 (PsyMedRP-v1-13B-p4): This phase involved merging The-Face-Of-Goonery/Huginn-13b-FP16 with a Gryphe gradient and the PsyMedRP-v1-13B-p3 model.

Finally, the Undi95/LimaRP-v3-120-Days model was applied at a 0.3 weight to PsyMedRP-v1-13B-p4 to create the final PsyMedRP-v1-13B model. This complex merging strategy suggests an effort to integrate diverse linguistic and potentially specialized capabilities from its constituent models. The model is currently undergoing testing, with a 20B parameter version planned for future release.