Llama-3.1-MedIT-SUN-8B is an experimental 8 billion parameter language model developed by MedIT Solutions, built upon Meta's Llama-3.1-8B-Instruct architecture. This model leverages proprietary MedIT-mesh techniques for model merging, combining components like arcee-ai/Llama-3.1-SuperNova-Lite. It serves primarily as a proof of concept for exploring model fusion methodologies and is intended for research and experimental purposes.
Loading preview...
Overview
Llama-3.1-MedIT-SUN-8B is an experimental 8 billion parameter language model developed by MedIT Solutions. It is built on Meta's Llama-3.1-8B-Instruct architecture and explores model merging techniques, specifically combining arcee-ai/Llama-3.1-SuperNova-Lite with the base Llama-3.1-8B-Instruct model.
Key Features
- Base Architecture: Meta's Llama-3.1-8B-Instruct
- Parameter Count: 8 billion
- Development: Created by MedIT Solutions
- Merging Technique: Utilizes the proprietary MedIT-mesh technique for model fusion.
Purpose and Usage
This model was developed primarily for testing and research purposes, serving as a proof of concept for exploring the potential of model merging in language model development. It is considered an experimental release and is recommended for research and experimental use only, not for production-ready applications. Evaluation results on the Open LLM Leaderboard show an average score of 30.04, with specific metrics like IFEval (0-Shot) at 78.37 and MMLU-PRO (5-shot) at 32.40.