Nexesenex/Llama_3.2_1b_RandomLego_RP_R1_0.1
Nexesenex/Llama_3.2_1b_RandomLego_RP_R1_0.1 is a 1 billion parameter language model, merged using the Model Stock method from a Llama 3.2 base and two other Nexesenex models. This merge focuses on combining existing models to create a new iteration. It achieves an average score of 12.81 on the Open LLM Leaderboard, with specific scores for IFEval (55.43) and BBH (7.94).
Loading preview...
Model Overview
Nexesenex/Llama_3.2_1b_RandomLego_RP_R1_0.1 is a 1 billion parameter language model created by Nexesenex through a merge process. This model was constructed using the Model Stock merge method, leveraging mergekit to combine pre-trained language models. The base model for this merge was KidIkaros/Llama-3.2-1B-Instruct-abliterated.
Merge Details
This model integrates components from two specific models:
Nexesenex/Llama_3.2_1b_Dolto_0.1Nexesenex/Llama_3.2_1b_OpenTree_R1_0.1
Each of these models contributed with an equal weight of 1.0 during the merging process, which was configured to use bfloat16 data type and normalize the weights.
Performance Metrics
Evaluations on the Open LLM Leaderboard indicate an average score of 12.81. Key performance highlights include:
- IFEval (0-Shot): 55.43
- BBH (3-Shot): 7.94
- MATH Lvl 5 (4-Shot): 5.66
- MMLU-PRO (5-shot): 6.26
Detailed evaluation results are available here.